Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console

114 points - today at 6:32 PM

Source

Comments

hypfer today at 9:29 PM
Kinda cool but also entirely useless given the limited VRAM.

The readme oversells it by pretending it would be something actually usable, when instead it is a cool achievement to make it happen.

xnx today at 8:26 PM
I've never even considered getting a PS5, but hackers converting a locked-down appliance into a general purpose computer has me really considering.
shiftpgdn today at 8:14 PM
Conceivably this should also work on the BC250 cards, which are ex-cryptocurrency mining motherboard built around AMD's "Cyan Skillfish" APU which is a cut-down version of the PlayStation 5's custom silicon. You can still pick them up on ebay for $150-$200.
Rabbit504030201 today at 8:05 PM
The project's primary site should not be GitHub, and I am serious. Playstation 5 is not some historical curiosity and Sony might strike the page down any hour now.

Those projects should default to hosts less geared toward corporate governancy; custom internet domain being the first requirement.

pwython today at 7:57 PM
With this in mind, having LLM-powered NPCs would be a beautiful thing. There's been limited testing on PC games (PUBG Ally), but I can imagine it will be the standard.
kevmo314 today at 7:04 PM
Not long until Anthropic's dreams of self-replicating models starts taking over PS5s!
randomblock1 today at 7:15 PM
drewg123 today at 8:04 PM
For somebody who hasn't kept up with all the different open source models and what token rates mean, how would using this compare to something like codex or claude? I assume it would be much slower and less capable..?
deleted today at 7:07 PM
Kurtz79 today at 7:07 PM
Don't give Sony ideas...
ThrowawayTestr today at 7:02 PM
11.4G of shared memory. Could do something interesting if you don't have a GPU.
armcat today at 7:29 PM
The toaster is next!