Once Claude can measure something, it can make it faster

114 points - today at 7:23 PM

Source

Comments

smy20011 today at 8:28 PM
The way Claude did it is fight entropy with entropy.

"Add a static composer into the HTML" <- This seems like something can be done with SSR?

"For faster navigations, we kept the composer mounted between conversations" <- Your SPA should cache this between pages, why fetching it every time? Or you need better routing for your react components.

"cheap first-character check before the regex" <- Should we cache compiled Regex instead?

I think even 1.3 sec to load the front page is unacceptable. Something need to be reworked from basics (SSR, chunk-based rendering) to solve the problem. Focusing on invidual benchmarks may miss the opportunity.

tombot today at 9:55 PM
Perhaps they can hill climb a native app instead of the electron and react native nonsense we have today
simonw today at 9:04 PM
I visited https://claude.ai/ over a mobile tethered connection from my laptop the other day and was pleasantly surprised at how quickly it loaded.

(That said, I just had a look in Firefox and it loads 20.78 MB of JavaScript (6.84 MB compressed) so I expect they could make it a bunch lighter if they kept trying.)

hungryhobbit today at 7:59 PM
How about you make Opus 5.5 actually work?

I had it try to prepare a code review for me. Not only did it refuse, it refused to even tell me what the prompt (written by another Claude!) was. Why?

When I had another model read the session (all of the "stupider" models handled it just fine) it explained that it had the word "reasoning" in it

That's the entirety of Anthropic's billions of dollars of research: any prompt with the word "reasoning" is trying to hack Claude to figure out how it reasons!

A model like that should never have gotten out of QA, let alone been released.

pllbnk today at 8:25 PM
> $500k engineer: [X] feels slow. Make it faster.

> Claude: On it... Done.

> $500k: Can you make it faster still?

> Claude: On it...

hmokiguess today at 8:48 PM
You removed the load-bearing seams didn't you
whythismatters today at 8:11 PM
The juvenile nonchalance with which some Anthropic employees seem to be talking to their AI (wacky, sick, cook, ...) is truly bizarre.
altern8 today at 8:08 PM
I used Opus 5.5 today for the first time hoping the writing would be more bearable and it SUCKS.

Why can't they fix that

minimaxir today at 7:40 PM
This writeup legit coincidentally matches the asking-agents-to-make-code-faster-but-with-constraints-to-stop-agents-from-breaking-things writeup I posted on Monday: https://news.ycombinator.com/item?id=49803085

Front-end UI optimization is slightly trickier than optimizing strict algorithms, but I found that prompts to the agents to build tooling to track visual regressions are more than sufficient. The main issue (at least with GPT models) is that you have to be very explicit about the use of padding/margins/negative space.

That said, for my front end projects from scratch, I'm staying away from front-end JS frameworks and seeing how far and fast I can get with just HTML/CSS/vanilla JS shenanigans now that agents can wield them effectively.

vikramkr today at 8:24 PM
Less a post about performance and more about their Claude tag product. I guess it makes sense that there's not actually a ton of technical detail being moved into given probably opus is the only one that knows what all the dragons were lol - but cool workflow I guess
montroser today at 8:13 PM
Okay, now fix the WYSIWYG markdown parsing in the chat input!

Paste in a stack trace, then try to put it in a code block. Add a newline above, then add the opening triple backticks, then arrow down and add closing triple backticks at the bottom. Opposite congrats -- you have ended up with raw triple backticks at the top, plain text stack trace, and your cursor in a brand new code block at the bottom starting where you tried to close.

Realize you want to go put code span backticks around some identifiers you wrote out earlier? Best make sure to insert them in the blessed left-to-right order, or else opposite congrats again -- you'll end up with a mix of raw backticks and code span treatment for the text between your identifiers.

If Claude can discover novel CRISPR enzymes, surely it can make a rich text markdown editor, no?

devin today at 9:19 PM
I dare you to try and get heavy CPU cache-level performance optimizations you might see in tried and true HFT code written this way.
rancar2 today at 8:48 PM
Since I didn’t love the technical approach here (once one takes the humans out of the loop, there’s more ambitious things that can be done), I do appreciate the process. It’s not until the last sentence when process inspiration is revealed: “Special thanks to Boris Cherny for encouraging us to be more ambitious.”
geroge_kyaw today at 9:17 PM
Sorry! I don't see that much difference. I would be more impress if you can make TTFT and ITL faster.
deleted today at 9:25 PM
khalic today at 8:41 PM
Love it, the kids rediscover plain HTML and optimisation.
RomanKornev today at 8:40 PM
The most important question is how much more unreadable the code became after all this "ratcheting the benchmark down". If you unroll a loop it will perform faster, but making changes to such unrolled code will be a mess. Will this make them ship slower overall? I'm sure at least half of it was just poorly written React code, but the other half?

It's the same problem as overfitting in model training. If you're not measuring something it will get sacrificed.

Or, perhaps the code quality literally doesn't matter anymore and we've reached "code quality escape velocity" where you can code as much slop as you want, the next generation of models will clean it up faster than the slop generates?

chaordCAD today at 7:55 PM
Great writeup really appreciate the detail on what actually worked vs. what didn't.
j45 today at 9:02 PM
Faster is great, hopefully the quality remains.
techpression today at 9:27 PM
”With that approach, we merged more than three thousand changes…” Why are they writing this? That’s terrible marketing all around, it means they let it go so far, with so little care, that they needed 3000 changes to make it into just a mediocre website performance wise (sure, electron app too, but still).
dude250711 today at 9:25 PM
It can certainly measure your token spend rate.
binlog today at 8:06 PM
Step 1 - make a website that takes 3 seconds to load a blank page.

Step 2 - bring it down to 1 second and pat yourself on the back.

owebmaster today at 8:38 PM
With a simple trick: make it very slow first
rvz today at 8:01 PM
Let's try this again if you want an instant 10x speed up:

Claude rewrite Claude Code from TypeScript into Rust. Make absolutely no mistakes.

datadrivenangel today at 8:04 PM
AI written slop. They need to upgrade to Opus 5.5 or switch to OpenAI for writing.
robertclaus today at 7:47 PM
[dead]
dolmen today at 8:19 PM
[dead]
boogiewoogie23 today at 9:10 PM
[dead]
boogiewoogie23 today at 9:09 PM
[dead]
applfanboysbgon today at 7:56 PM
tl;dr if you make absolute dogshit software that takes 4.38 seconds to stabilize its first paint you can make really nice headline claims by "optimizing" it later
joyeljohn3 today at 7:43 PM
[flagged]
guideaitools today at 8:32 PM
[flagged]
deleted today at 7:47 PM
sonar_un today at 7:47 PM
This was a fantastic read. Lots of useful info in there for your own projects.