Some thoughts about Anthropic's new cryptanalysis results

68 points - today at 4:42 PM

Source

Comments

simonw today at 6:20 PM
This is good:

> If you’re under the impression that these models are ā€œglorified autocompleteā€ or that progress is slowing down, I need to urge you: stop thinking that. The models are very intelligent and capable, they are getting better at a fast clip. I can cite measurable and impressive progress over just the past five months on specific types of problem I’ve asked them to look at. [...]

> On the other hand: if you think that models are super-intelligent or that AGI is already here, you should also stop thinking that. Working with these tools is like swimming in a pond where the ground drops off sharply. One minute you’re wading comfortably and there’s support under your feet. Then suddenly you cross a specific line, and you’re back to swimming on your own.

simonw today at 6:11 PM
> both outputs of Claude Mythos, their (still) unreleased advanced model

That sentence gives the impression that Mythos might be released in the future. That's clearly not going to happen - it's already "released" in as much as selected, trusted partners can access it, and the rest of us get it in the form of Fable - which is Mythos but with filters that downgrade you if you try to use it for anything even remotely related to cybersecurity or biology.

(The other day Fable 5 downgraded me to Opus after I asked it to explain the difference between tusks and teeth.)

john_strinlai today at 6:07 PM
>They [anthropic] appear to have just told it to get some results and then strapped its nose to the grindstone until it found some.

it is fun how well this works.

i cant find the link immediately (will look and edit with it), but somewhere in the "hello there the jacobian conjecture is false thanx" thread, someone brought up a different conjecture breakthrough where the prompts were basically just repeated "no, keep going" until a result was found.

edit: https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...

i especially like "you should do a breakthrough". each prompt is less than ~20 words. makes me really question the whole "prompt engineering" stuff.

throawayonthe today at 6:13 PM
> I asked Claude for its thoughts, and it doesn’t mince words: ā€œwhat makes this genuinely interesting — and, frankly, a little embarrassing for the field — is that none of the ingredients are exotic.ā€ The TL;DR is that someone just did a much more thorough job applying all of our known tools. In short: the sort of things that attack AIs are wonderful at.

did i just read two summaries/TLDRs (in a row) of the already-two-sentence summary right above?

deleted today at 6:51 PM