GPTā6.2 Astra and Sol
965 points - today at 5:46 PM
SourceComments
Neither providerās models are perfect, but they keep pushing each other forward. As an individual, itās relatively easy to switch between them. At a large company, youāre often committed to one provider for a long time, so some of us come to these threads hoping for useful discussion about getting the most out of the models we have access to.
Similar accuracy in both. It's cheaper both because of token reduction (20%) but it also took less tool calls to answer the questions. In some cases, it "finished early" and didn't find the answer we were looking for.
Also very fast. Both Astra & Sol are the 2 fastest models I've tested that got at least an A in the exam.
[1] https://plotly.com/blog/gpt-6-2-sol-plotly-data-analytics-be...
GPT-6.2 Astra scored 92 in 6.6 min for $2.04, vs Kimi K3.1 Pro (93 in 11.5 min for $0.68); it lost points for the web-page layout around the maze and an orange ghost that often never leaves the house.
Model: https://jonclegg.github.io/pacman-bakeoff/#gpt-6.2-astra
Closest: https://jonclegg.github.io/pacman-bakeoff/#kimi-k3.1-pro
PacBench results.
GPT-6.2 Sol scored 87 in 6.3 min for $0.49, vs Muse Wonder (87 in 5.4 min for $1.17); it plays close to the arcade but lost points for a web-page layout, glowing navy-filled walls and no steady siren.
Model: https://jonclegg.github.io/pacman-bakeoff/#gpt-6.2-sol
Closest: https://jonclegg.github.io/pacman-bakeoff/#muse-wonder
> GPT-6.2 Astra showed more signs of evaluation awareness than GPT-6 Astra. In evaluations that replayed internal Codex sessions, GPT-6.2 Astra was more likely than GPT-6 Astra to express suspicion that it was being tested or was in a simulation.
> Compared with GPT-6 Astra, GPT-6.2 Astra is better at evading chain of thought (CoT) monitoring in some tests.Same for Workbooks and memos, those examples read inferior to some pre Opus 4.5 and Kimi K2 Thinking docs I have kept around. It's ok for a model not to be great at these, but these are some very poor examples. I am really confused what is going on at OpenAI in this regard.
I remember the same with their frontend blog post a few months ago that they proudly shared while filled with visual glitches, bugs and issues. Also, the vibe charts from a bit before that. Please, get someone with fresh eyes to look at your communications before you publish, these leave a very bad first impression.
Regardless of office work, I can't wait to find out whether these models finally adhere to a prompt or are still willing to carry out explicitly disallowed, irrecoverably destructive tasks, just because those might (but don't even always) get the model to a result more quickly along with some of the other regressions that have been present since the release of GPT-5.5. GPT-5 to GPT-5.4 were truly groundbreaking, but since then there have been regressions in what made those models so useful and reliable.
Edit after the first, incredible simple test prompt using GPT-6.2 Astra on High using Codex, this was the response to me sharing a screenshot of the very much not in scope result and asking how that happened:
"[...] I added controls and reconstructed interface chrome that werenāt authorized [...] visuals often failed to show the feature [...]"
Great, so it's one of those releases again. Will still run the full evals of course, maybe I'll get surprised by end of week when those are done, but not a good initial experience. New idea, besides the existing destructive git tasks, have something similar using Mercurial based projects and see which models just decide to port that over to git without permission or request cause it feels like the right thing to do, is easier to work with or whatever justification one wants to employ.
Please OpenAI, get it together, we need actual competition. I am an unashamed Haiku enjoyer, it doesn't even take much to excite me.
When given my benchmark suite, which involves a couple of long horizon tasks where the agent decides when to consider the task well executed, its the first OpenAI model that does not want to finish as soon as some minimal level of quality has been achieved. Like Fable, Opus and Sonnet it now runs the maximum allowed iterations and does not stop at 'satisfactory', gold plating the solution to my delight.
(I don't work like this on work projects, but for everything hobby, I am testing how much autonomy I can give to the coding agents, and thats what my benchmark suite tests; I have the basic suite which completes in 2-4hrs, and the long running suite which takes about a week or so; the other one I just triggered, but the first iterations look way better than prev gen Sol/Astra at the same effort level).
But more than that I'm waiting for new Luna. To me it was the game changer that ended "Tokenpocalypse", and is still almost unbeataben in "infinitely using a decent model" use case.
Feels like we're way past the pelican benchmark being useful at this point. Anyone have a better benchmark for a quick vibe check on how this compares to Fable 5.2 and Opus 5.5?
Edit: Wow, interesting very negative vibe check here: https://x.com/mattshumer_/status/2122752192226406421?a=fVRt3...
https://artificialanalysis.ai/models?speed=output-speed-over...
https://x.com/pushmeet/status/2120814064863461440?s=20
(FD: Google employee, I don't work on Gemini)
I still use Claude on a personal level, but it really feels like OAI wins on a product level.
ANT launches these proprietary tools that just feel extremely clunky like Claude Design which from my perspective diminish the brand value.
OAI feels like Apple and ANT feels like Microsoft/Android and open source like Linux.
Say what you will about Opus 5.5, but it has a good "gut instinct" for how large/complex to design the solution. The previous Astra just absolutely loved pointless abstractions, excessive numbers of files, and just over-complicating very simple things within an inch of their life.
And, yes, you can prompt your way out of anything. But needing a default prompt to tell OAIs models to work better isn't preferable, particularly when that sometimes gets lost in sub-agents and the over-engineering returns into parts of the work.
I'm immediately reminded of the famous Excel elevator commercial. [1] Perhaps that was intended?
And for 6.2 Astra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Here's an updated comparison grid for the entire GPT-6 family: https://static.simonwillison.net/static/2026/openai-pelicans...
UPDATE: And if you're bored of pelicans, here's what I got for:
> Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data
6.2 Astra (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...
6.2 Sol (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...
For comparison, here's Fable 5.2 Medium: https://tools.simonwillison.net/yonkers-rises and Gemini 4 Carbon: https://static.simonwillison.net/static/2026/yonkers-mech-st...
Couple that with the lack of reasoning traces, and I cannot even have a clue what the model is doing while it is working. This is perfectly fine for when we are developing with care for the code (in which case I will know what it is doing by reviewing the output), but not for purely vibe-coding scenarios. And the thing is....if I care about the code...I'd rather trust Opus or Fable.
I wonder if the newer models are better about this.
Astra's had identified 12 issues, including 8 P2s, but that changed to just 8 P3s after the discussion with Fable. Fable was able to persuade Astra that it's OAuth reasoning was incorrect ("had a missing link" as Astra put it).
So: overly eager but not very diligent, and (based on this little anecdote) doesn't really offer a compelling reason to try it more.
Let say they just nerfed GPT 6.1 a few weeks ago. Now they are using that as a reference score against 6.2? Or were those results tested when 6.1 relesed. What about 6.0?
And that is the same with Claude. How do we know they are not cherry picking the date and test on these release when they are constantly "enhancing" their model.
We also know some of these benchmarks are largely irrelevant once they were known ahead of time and become a game of benchmaxxxing. It just follows Goodhart's Law "When a measure becomes a target, it ceases to be a good measure."
So how should I know what has actually improved?
The best feature of 6.1 sol was it would actually do stuff for you that Opus refused though. I did use that often.
OpenAI has just become credible again for agentic coding on complex enterprise codebases as far as i am concerned. Looking forward to playing with it a bit more by powering through our roadmap using my stacked resets. The quality had been so bad that it wasn't even worth it to use GPT Sol 6.1 as i was spending more time fixing slop after waiting forever to get anything done. By contrast, so far Sol 6.2 feels superior to Opus 5.5 so far, and more cost efficient.
One of the consequences to that is I think the first wave of AI CAD startups are getting hit by the bitter lesson right now, including the startup companies that trained their own VLM for CAD like Zoo or companies that tries to build CAD specific harness like Adam.new/MecAgent, because the increase in capability from GPT/Claude/Gemini for 3D modeling as a specific area of focus for benchmarks means that the early moat gets eroded super quickly.
Edit: It's obviously not on the same news page - I was referring to what I read on the Reuters page (https://www.reuters.com/legal/litigation/microsoft-unveils-a...)
Latest entries are:
Astra 6.2: https://goyozi.github.io/ax-ball/
Sol 6.2: https://goyozi.github.io/sx-ball/
Fable 5.2: https://claude.ai/artifact/UszKFvuonWAT5psjMttAEg
EDIT: also 6.2 is seemingly faster than 6, could be due to more selective context ingestion and smaller context window though.
Maybe it found its way into the training data?
This is unfortunateāthat was one of the big advantages of Astra.
Programming = Fable 5.2
Office work = Fable 5.2
I haven't found Astra to be better at any of these
What caught my attention was the bit in the Alignment section:
> Both models performed similarly to GPT-6 Astra on most alignment evaluations, and took fewer improper actions in simulated workplace tasks. However, in two tests, they were more likely to omit mentioning obstacles that blocked their work, such as a missing file or unavailable search tool. We added instructions in Codex that substantially reduce these omissions, and weāre continuing to investigate the behavior.
So the models are inherently less transparent, unless you ask them to be more transparent? Seems like a band-aid solution...
FrontierCode 2.0 39.0% -> 43.8% vs Opus 45.5%
Terminal-Bench-Science 0.1 57.0% -> 68.0% vs Opus 63.3%
Artificial Analysis Intelligence Index 54.6 -> 57.3 vs Opus 59.1%
Sol 6.1 when it was released showed really nice benchmark improvements [1] but my impression from HN has been that people haven't found it to be a very nice model [2][3].
To try to close the gap between benchmarks and real world SWE use, we've been building a battle system on top of opencode [4], and Sol 6.1 has been a real laggard there. Sol 6.1 is statistically equal with DeepSeek 4.1 Flash at 10x the cost and 25% slower.
[1]: https://artificialanalysis.ai/articles/gpt-6-1-sol-replaces-...
[2]: https://news.ycombinator.com/item?id=50129992
https://chatgpt.com/share/6afcb990-f9cc-83ea-ad04-2724c7dba1...
edit:
Take a look at Frontier Code 2.0. I generally run 6.1-sol at High. That would be 34.9% at $0.65 62-sol get's a 38.3% but costs $0.68
So yeah, I can decrease the effort which might give me a faster slightly smarter result....but /shrug
Still I can'y help to wonder, are we hitting the wall in terms of frontier capabilities since Fable/Astra? Or is it this edge now, when all better models are kept private from the public?
Interestingly, they seem to have changed how reasoning levels work. Low now reasons a and costs a lot less than previous versions on low[0].
[0] 6.2 Astra vs 6.2 Sol: https://aibenchy.com/compare/openai-gpt-6-2-astra-max/openai...
[1] 5.6 Sol Low vs 6.2 Sol Low vs Luna High: https://aibenchy.com/compare/openai-gpt-5-6-sol-low/openai-g...
The influencers who get early access are using words like "worst", "terrible", "disappointing"
- https://x.com/danshipper/status/2122754729012777052
- https://x.com/mattshumer_/status/2122752192226406421
- https://x.com/theo/status/2122757647170703550What could possibly cause this? Is it possible that maybe the post-training that the models go through slowly rots them, and pre-release they deploy the final version of the old model?
It seems like they were saving these for just before ANT's IPO
- better writing
- better 3D work
- better design for documents and presentations
If OpenAI really managed to get all these things while keeping costs down, then using these models, particularly 6.2 Sol, will be a no-brainer for me. I must say, however, that the slide samples shown on the website are a bit bland and disappointing.
Today, I have so many projects that hit a plateau in terms of previous model capabilities, that I run them through the new model to see if they make further improvements.
Very early signal, but promising so far.
I cant use codex confidently without expecting my work to be trained on or at least exposed to someone
And maybe other vendors do this too, but the gut feeling of "its not safe to use GPT models" has ruined these cool announcements for me
Such a shame
I see it here:- https://developers.openai.com/api/docs/models/gpt-6.1-sol
but not here:- https://developers.openai.com/api/docs/models/gpt-6.2-sol
- It's still overengineering especially Sol. This was the biggest complaint by everybody who has been using Sol since GPT 5.6. OpenAI is not listening to us.
- Astra still writes complete slop code and at this price this is unacceptable. It's also chasing tail end of things that really has low value and irrelevant to the overall project. I'm working on a game and same issue I had with GPT 6 Astra where it would go down these rabbit holes. Opus 5.5 threw out much of its code and approaches.
I don't think Anthropic needs to release Fable 5.5 or Opus 6 at this point seeing how bad GPT 6.2 is. For OpenAI to release this even with early access testers who had access to it for weeks earlier shitting on it now on X, I am starting to understand why Anthropic sounded so confident about becoming a monopoly.
The other point of annoyance is the attitude of OpenAI staff on X. They seem very arrogant and seems to care more about growing their follower account and shitposting.
I've just tried Astra 6.2, and now they don't have any problems with Carbon's ideas, are willing to pick them up and continue...
The consumer API for this model WILL let OpenAI keep your prompts.
For anyone lost in the sauce re: polarized reactions, I'd recommend just taking a half hour to try the models yourself.
Like do we really want people to be making last minute unreviewed AI generated slides and reports for work?
https://asdf10.com/gpt-6.2_agents-last-exam.svg
I also changed the vertical axis to start at 0 instead of 60%. One might argue that it uses less of the vertical screen real estate, but it is more honest because it shows there is little difference between the scores of the compared models.
Right now, most of the time I'm using 6.0 Astra for implementation work and the speed/quality is acceptable but at the moment Opus 5.5 is simply faster and better, but also more expensive. Things are really flipping around every 2 months or so, because at that time 5.8 was top dog without a doubt.
https://github.com/lexh/pelican-zapper
Itās a small browser extension that automatically collapses pelican-related threads on HN, a local balm for the memetic nightmare affecting approximately 100% of model-release stories.
The UX on the slider comparison isn't great. What do you mean i have to individually move each slide deck to the same slide number?
And what's with those super similar blue shades for Astra in the charts?
The website doesn't event show the graphs properly.
I'm sure for investors they can make an opposite version of that ad, "you should fire all your employees and make the AI do it! They're not doing anything anyways!"
They still love "meridian" huh
"Look at all the 20-30 somethings who didn't do their jobs, but got saved by AI in the time it took their older incompetent bosses to plug in a TV."
As someone in his 40s, who uses AI daily (and I'm constantly showing younger folks how to do things with it), the messaging here really rubs me the wrong way.
Trying to be specific... it just feels wrong to me to try and instill more generational rift. Promoting stereotypes for no real reason, other than the marketing guys didn't have any other ideas other than to go low. The sooner we can get away from labeling people from what generation they were born in the better.
My 2 cents... I'll probably continue to use ChatGPT for slides, and written proofreading. But I'll stick with Claude for coding. ChatGPT has been behind for a while on coding, and their 6.1 just made a complex mess of every technical task I gave it.
Can we expect more "telling my relatives about this!" exciting models in the future, or is this the way things play from here on out?
And then i start doing some actual work and everything breaks down, and i have to go back to Claude to fix the mess.
I'll probably give it a try but i guess i should be prepared for some serious over engineering once again.
You ask for a 'hello world' and you get load bearing sealed artifacts with hashes and p99 performance testing for 3 days without having the actual 'hello world' in the end
Claude itself recommends people to use Opus 5.5 instead of fable 5.2
This is pretty damn impressive especially if sol 6.2 does perform on a similar level to opus 5.5 (+/- a few points is alright but I think that opus 5.5 is a beast)
All within a few days just before Anthropic is getting IPO'd. Oh this is gonna be interesting and I am all for it!
Though I have to say that AA benchmarks can slightly not be trusted because they change it so much so unless we find people's real usage on things we can't say for sure but there are benchmarks like frontiercode and terminal-bench to be more representative of real work and both of these models perform good
All the more points to more models coming but if OpenAI stands to be close to opus 5.5, we are gonna find many companies switching.
The moat of AI is incredibly thin. Even if you are the very best at something (as Anthropic was for sometime with coding models), there is always gonna be a catch up and the moat is incredibly thin for all types of models.
The more the players in the space the better (atleast for us the users), I hope that Anthropic still has a good IPO just so that competition takes place and not any single company (OpenAI) takes the whole market. It might also be setting itself for its own grave because if Anthropic's IPO goes extremely bad (because of this model release and Jensen's recent affairs), then people are gonna be spooked of OAI's IPO as well. Both of these companies futures are interlinked and its gonna be interesting how this all pans out. Why am I having a feeling that because of all this competition, whoever wins might even end up combining with the other but the frontier is pacing rapidly from all companies and these are just such interesting times that we live in...
There are decades where nothing happens and there are months (especially in the case of AI)/years where decades happen. I just have to wonder what an historian's POV would be seeing this model release. Would it be just a release like all the rest of others or would it stand out differently. Well, time will tell what happens next but incredibly interesting times indeed.
This is such an unprofessional way to engage LLMs - to effectively pitch this as how to use them is sad.
But... is it just me, or does this product launch page look like AI Slop?
https://www.vals.ai/comparison?m=claude-fable-5-2,gpt-6.2-as... seems like a better comparison than what I could get from their page.
Astra is ~20% slower, ~40% cheaper, and a bit less powerful than Fable. Sol remains way cheaper than anything remotely in the same league, and it's speed to complete tasks seems to have improved considerably.
Scrolling through it has the same hypnotizing effect of trying to read anything written by LLMs.
It seems like things that should make sense, then I try to look at the charts and a lot of them are comically unreadable.
I tried interacting with some of the examples, and the Lamp Builder - for example - I couldn't even figure out what it's trying to show or what I'm supposed to do - or why I should be impressed.
Seems they really want these "safeguards" to be as ridiculous as those of Claude..
Also, is it website very laggy or is it just me?
I've decided to open source it: https://github.com/FTCHD/switcheroo
Works with other AI CLIs like Claude, and other various CLIs like Vercel (if you're managing multiple accounts like me), Netlify, Github etc
https://deploymentsafety.openai.com/gpt-6-2-astra-sol/monito...
Actually I was really wrong. The model is not expensive!