Lot more details in the linked report https://ai.meta.com/static-resource/muse-spark-1-1-evaluatio...

From Terminal-bench-2.1 details,

> We use a bash-tool-only agent harness to evaluate 89 Terminal-Bench 2.1 tasks from the official repository, where resources are capped at 6 CPU cores and 8GB RAM.

This disqualifies the results. Each terminal bench task has a cpu upper limit and RAM upper limit. Overriding either is disqualification.

For reference, in tbench-2.1,

1. 0 out of 89 task allow 6 cpu cores (highest is 4, and i think only 1 task)

2. 8 out of 89 tasks allow 8GB RAM

This kind of shady benchmarking (I was talking about it just yesterday in a different context https://news.ycombinator.com/item?id=48838212) takes all joy out of building a harness to improve benchmark performance of a model because no matter what you do, you won't beat the headline (cheating) number. This is presumably why this model is not in the official benchmark leaderboard https://www.tbench.ai/leaderboard/terminal-bench/2.1

As an ex Meta employee, this is a little sad but not massively surprising. 'Number go up' is the core performance evaluation metric until PSC is done and you move on.

I had a few days of preview access, which was long enough to put together a plugin for LLM. You can try the model out in the terminal like this:

  uv tool install llm
  llm install llm-meta-ai
  llm keys set meta-ai
  # paste API key here
  llm -m meta-ai/muse-spark-1.1 "Generate an SVG of a pelican riding a bicycle"

Here's the result: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

For comparison, here's the pelican I got from Muse Spark 1: https://simonwillison.net/2026/Apr/8/muse-spark/

Maybe Zuck should double down on his "spoiler" role with models rather than compete head-to-head.

He doesn't have to match Anthropic or OpenAI model revenue if he can deflate theirs by 99%.

All he has to do is keep spending a few billion dollars developing frontier models, release them as open weights, and turn coding models into a commodity. He also needs a good OSS reference harness to match. Very few people are in a position to do this and for it to make business sense.

That's quite likely where things are headed regardless, and he could speed it up significantly.

We should all hope models move from proprietary products to commodities the way compilers did.

This may be one of the best things Zuck could do for the world.

Since it was making rounds yesterday here is what muse generated for the 3d rubik's cube prompt - https://ashish01.github.io/rubik-muse/

The pricing is insane: $1.25/$4.5 for 1M tokens, and $0.15 for cached input!

https://dev.meta.ai/docs/getting-started/pricing-rate-limits

Interesting how the prevalent opinion until yesterday seems to have been that OpenAI & Anthropic are irreversibly ahead and now with xAI and Meta at least delivered something that's competitive with useful models and cheap too. Granted, the narrative that the two leading labs are ahead still holds with Fable (and perhaps an upcoming GPT6), but it's not as over as common knowledge by the opinion leaders would have us believe.

I personally do not like Meta, but I'll say this. The more competition, the better for regular consumers. (Enterprise too)

- Chinese models

- Grok

- Meta

- Google

- OpenAI

- Anthropic

I think this is a win. I'm building like crazy to take advantage of all these subsidized tokens while I can.

How is every company able to show itself at the top of every benchmark?

Their published benchmarks seem to indicate that it's pretty good at coding and multimodal, but VERY good at successful tool calls.

What kind of use case would be best for that shape?

It seems to trade blows with GPT 5.5 and Opus 4.8 in performance while being cheaper than GLM 5.2.

> Model API is not available in your region.

:(

Well, Vietnam is not in the list of restricted territories.

Anyway, what is "your region" ?

Is this where I am now, or is it where I activated my Oculus 2 five years ago ?

Glad to see Meta back on track! Users will benefit greatly from this competition.

I missed the fact that Meta was developing and releasing closed-weights models... bummer. Would be great to see some more progress with American open-weights models.

Very strong pricing, cheaper than Grok 4.5, particularly the cached reads. We'll have to wait to see if it's actually worth using (it's not on OpenRouter yet).

I'm still confused is it available to public via some sort of subscription?

Why are the plans and pricing for all these products so complicated.

I don't know where I need to sign up to try it out. What is pricing? Is it API or subscription, what?

I had the exact same experience with Grok 4.5 as well.

Yeah, no thanks. I cannot think of a worse company to trust with additional personal data.

Competition for cheaper and efficient models is a good thing, regardless of if you don't like SpaceX, Meta, etc. Especially from US based labs

I for one am really glad to get competitive models that will push the major labs to bring prices down. While Chinese open source labs are also great, unfortunately when it comes to US/Western political pressure it won't often have as much of a bearing on labs bringing prices down, especially for enterprises.

Also if these numbers are true, this is truly breaking ground finally for Meta.

Good to see Meta finally back to releasing something at least worth evaluating. And it sounds like they did at least a bit skate to where the puck is going by focusing on tool and computer use.

How are people trying this? I don't see it on openrouter. Any ways of testing this without subscribing to meta stuff?

Considering the DeepSWE result (imho if you're gonna give value to benchmarks this is one of the best) it's not good enough.

Everyone has been loving to shit on the Alexander Wang acquisition but this seems legitimately impressive to me?

Meta's AI org when from a total mismanaged dumpster fire for multiple years to delivering a competitive model in less than a year on essentially their first try?

Interesting that neither meta nor xai chose to do open source given that they are both clearly behind Google, OpenAI and anthropic - and a serious us open source offering would give them a clear foothold.

My trust factor is gone with Meta right now. Has there been any independent analysis to confirm they didn't cheat on benchmarks again?

Not opensource.

Is this the model trained on Meta "draftees"? Are we seeing this in the jump on JobBench?

Tried to get access to the API, apparently the model API is not available in my region...

I have questions regarding if I should even care but I don't so Meta please keep enjoying the irrelevance. lmao

Haha their demo is AI spamming restaurants on Instagram. This is going to go really well.

A lot of these benchmarks are unfamiliar. Are labs just choosing the ones that make them look best?

This is not open-weights, right?

Meta is back in the game, albeit not at the top. Impressive stuff, nonetheless.

[dead]