It's the memory latency, stupid

194 points - yesterday at 6:38 PM

Comments

boricj yesterday at 7:38 PM

I work with another team at work that uses lots of message passing, C++ virtual methods in deeply nested class hierarchies, large DOMs and off-the-shelf C++ STL containers... on a STM32MP1, which has an in-order Cortex-A7 and a pittance of cache.

On the other hand, in my team I wrote my custom C++ STL containers that allocate inline, SAX-style processing libraries and an object pool with links that use index+generations like 2000s video games... on a STM32H5 with less than a megabyte of RAM.

Mechanical sympathy feels like a lost art to people I demonstrate it to... and I had to demonstrate to the first team with benchmark numbers that I would not be the bottleneck between us by orders of magnitudes.

swiftcoder today at 9:26 AM

I remember have a very long argument with a team I worked on, who were mostly convinced our service couldn't handle requests any faster because the CPUs were already at 100%. In the end we tripled the requests-per-second by cleaning up the service's memory access patterns...

bugake today at 7:59 AM

Modern CPUs have hundreds of megabytes of cache. That's more than the entire hard disk space required by Windows in the 1990s.

You could install an entire Windows system, emulated in cache of a modern CPU, including the hard disk, and modern computers are still slow, partly because the CPU does nothing while waiting for RAM to fetch data, and partly because decades of people prioritizing abstractions over efficient coding have made software totally inefficient.

brcmthrowaway yesterday at 9:39 PM

https://people.freebsd.org/~lstewart/articles/cpumemory.pdf

Has anyone rewritten this for 2026?

drivebyhooting yesterday at 10:44 PM

I feel like the solution presented is just a clustered hashmap or “locality sensitive” if you’d like it.

Still not orders of magnitude better.

quotemstr today at 1:11 AM

Uh, isn't this just radix-partitioned aggregation? Hierarchical (fixed two level) aggregation is old. What's novel here?

Microcosm of Pivot itself I guess. I just don't get why this project exists when DuckDB and Clickhouse both exist.

zittturkkpon today at 12:21 AM

[flagged]

deleted today at 12:49 AM

happyPersonR today at 12:02 AM

Shout out to the author, it seems coherent.

I like flamegraphs

I know folks use perf to help generate them

But in the future, now that Llm is here, get and agent to check assembly, PMUs and also see what you’re doing in assembly.

My 2c and happy coding to everyone :)