Coincidentally I've been playing with small (~30B) self-hostable models for coding tasks today -- specifically plugging them into Cloudflare OS (which I work on) and asking each to build a collaborative whiteboard.

I'm finding that the Mixture-of-Experts (MoE) models (Qwen 3.6-35B, and Nemotron 3.5 Lightning) are, well, terrible at this. They just couldn't get the job done at all. Went way off the rails. They are really fast though!

Whereas ~30B dense models (not MoE) are pretty decent. I tried Muse Glimmer, Gemma 4-31B, Qwen 3.6-27B, and Laguna XS[0]. They were all able to build a working collaborative whiteboard app, without any guidance (other than feeding back error logs to the model). I also asked each to then draw a monkey by calling the API of the whiteboard it has just built. Laguna drew random scribbles but the rest all managed to produce something monkey-like.

(Frontier models in comparison will write the app in one shot with no errors at all.)

Note that both Qwen 3.6 and Gemma 4 each have both MoE and dense variants. I find this very confusing, because e.g. ollama's model index typically only distinguishes variants by their size, but MoE vs. dense makes a huge difference in how they actually perform. IMO they should use a suffix, like Qwen 3.6-moe vs. Qwen 3.6-dense, or maybe Qwen 3.6-fast vs. Qwen 3.6-smart...

[0] EDIT: Turns out Laguna XS is MoE, I misunderstood. It performed similarly to the dense models. But maybe this explains why it couldn't write code and think about monkey shapes at the same time!

One major consequence of the ramapocalypse, I think, is an even higher focus on small efficient models. I personally believe that the multi-trillion parameter models are fundamentally missing things and the push to smaller, more efficient will drive evolutionary structural changes that will lead to future gains

> NeMo Switchyard, an open source library for smart routing

> When deployed, NeMo Switchyard can intelligently direct each request to the most capable and suitable model for the job

How do routers like this handle prompt caching when you send the second request?

Sticky models per session? but then the second message of that session won't be sent to a suitable model, and will only be sent to the same model as previous one.

They conveniently decided not to include the Qwen range of models in the Artificial Analysis graph, except the out-of-league Max variant. At least be brave and honest.

I would like to propose something:

- problem: massive deluge of information because of AI

- solution: human beings should adopt a minimalist style of communicating in writing.

- e.g. this entire website page can be ten bullet points.

24 comments so far about Nemotron on this earlier submission: https://news.ycombinator.com/item?id=49257947

I love the wave of new small model releases. Pleasantly surprising that an NVIDIA model runs so well on Apple Silicon using MLX! I was using nemotron-3.5-lightning:30b-mlx with OpenCode on my old (cheap) Mac this morning and no bad experiences except for running slowly.

big week for open models... seems like companies are noticing the 26-35b sweet spot... though I think a 12b-a1b-MoE model would be helpful for the 16gb folks

An open source model from Nvidia is a free drug to later buy their chips.

Nemotron 3.5 Lightning runs on how little GPU vram? Can q4 run on 16gb?

Good luck getting it to run with NVFP4 on a DGX Spark, the very architecture Nvidia created that format for.

[dead]

The new Meta 30B models seems A LOT better:

https://aibenchy.com/compare/meta-muse-glimmer-30b-xhigh/nvi...