Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat.

Just writing this down so I can be praised/mocked in 5 years.

Apple could probably sell a machine starting at $10,000 if they architected it as the sole place one’s Private Cloud Compute [1] ran.

It would need a path to a $2,500 machine, I think. But this is a niche I don’t think another consumer-facing brand could do like Apple.

[1] https://security.apple.com/blog/private-cloud-compute/

What I’m not sure to understand is that if you want to just run Claude code or openclaw type software with llm apis or subscriptions (and not run local models) to benefit from a local file system and always-on capability for ‘second brain’ type of workflows, I guess you don’t need a Mac mini but can run it on a raspberry pi or an old laptop ? Does anyone have experience with that ?

If I had the capital I’d make an household inference appliance.

No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.

Even the appliance faceplate would be a display showing stats like an old HiFi stereo.

Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory

Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind.

Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.

It’s not for the AI inference, it’s for the tool calls and desktop GUI app workloads and browser. There aren’t any on-device models capable enough of real work that can run on lower end Mac Minis. But for running a few browsers and GUI apps, you’re much better off buying a Mac Mini than paying for a more expensive and worse-performing container in the cloud. Browsers were not designed to run in Linux containers but they run optimally on baremetal desktop OSes. An M4 Mac Mini beats the single core performance of any VM you might rent in the cloud, in terms of raw compute per dollar (Geekbench scores).

> Apple's Mac mini and Mac Studio have become the machines of choice for running AI agents, according to Doug Brooks, Apple's senior product manager of Apple silicon.

This is mostly an US phenomenon, no Mac mini nor Mac Studio around here.

Only Thinkpads and Macbooks laptops talking to hyperscalers.

People bought diesel cars as miles per gallon was higher than petrol.

People are buying apple unified as electricity costs in many countries are very high, so cheaper to run than Nvidia setup.

As non-apple unified memory options increase, many people will have more choose those

When you don't limit on-device AI to "can be used to run coding model", Mac minis are great. My M4/16gb been working on a long term research project using Qwen 2.5 8b for months now. The performance is good enough for processing a lot of small text prompts.

This is a paraphrase of https://www.thedeepview.com/articles/how-apple-s-decade-long...

Being a Mac switcher since 2003 I am as much of a fanboy as anybody else but this quote from the article caught my attention, and smells like PR.

> Many AI tools are also Mac-first or Mac-only

I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?

Except that Mac ultra M3 they talk about is now only being sold in the 96 GB configuration. It’s no longer being sold in larger Ram configurations by Apple Apple because of the global RAM shortage. And you can not add memory after purchase because it’s integrated/soldered on.

Now just imagine they'd kept making the Mac Pro and enabled compute offload to GPUs. Or even just passthrough to Linux VMs. Would've been quite the AI machine.

Apple could dominate this niche if they decided, for a while until prices fall, to eat some margin and bump up RAM in high end models. Couple that with a new M series chip with even faster AI performance.

It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists.

For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.

I don’t understand why they are not reintroducing the Apple Xserve?

Apple doesn’t have an ads business, and it is inevitable that Google/openai/Anthropic are going to seek to monetise consumer ai via ads.

So the ad free Apple on device experience will be welcome.

> "The speed of AI development right now is just crazy," Brooks said. "I can't imagine where we're going to be a year from now, three months from now, or even a month from now," he added.

I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI.

Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.

Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.

The issue with the Apple is that they didn’t really develop any competitive local AI machine. Their strategy/marketing falls flat when you ask them how exactly they implement AI: they buy it from Google cloud. In the future local AI may become a thing but that’s 4-5 years away. I count the 2q-4q and atrocious performance as “local ai” only for the enthusiasts crowd not for people doing competitive work.

Oh please the neural engine is mostly useless for LLMs. Siri in iOS 27 is laughably pathetic and slow compared to GPT Live DESPITE sending personal context to their (attested) cloud to execute anything but the most basic queries. Still years behind.

Ok, I didn’t want to take the bait but this one’s just too much.

> “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.”

Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!”

<neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac!

> “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.”

<neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone!

How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens.

Was that so hard?

> "people often want a system that's under their control, isolated from their primary machine, and capable of running 24 hours a day, seven days a week," said Brooks. "A Mac mini is an amazing system for that," he added.

These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing.

Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.