Building an (almost) fully self-hosted, sandboxed, agentic software factory
67 points - today at 4:27 PM
SourceComments
Wondering what the actual verification loop looks like once you start taking these systems to production.
I'm interested in hearing from folks who are hosting their own GPU to run coding models. So far my own results are... not great. Seems like frontier models are needed via the big providers?
I pointed Claude at those for hours and hours, didn't fucking work, ended up creating more and more slop. Until I stepped in and fixed it in a few lines of code. Thankfully with all my AI assisted work, I took time to understand everything so I was able to debug and fix.
The problem for software factory is, that up to a certain point, you'll encounter some nasty bugs that are really important for your features, but you can't ship it because while the AI deals with the 90%, turns out that last remaining 10% are the one that make or break your project. And you need to deal with that remaining 10%. But how can you deal with the remaining 10% if you don't understand that 90% or majority of it?
Also, don't listen to influencers like Kent Dodds or Uncle Bob or Steve Yegge, who advocated for software factory walk away from the code kinda thing. They are already rich, and they sell teaching content, but most importantly, their time in this industry has passed. They are not in the weeds doing enterprise software development anymore.
Their first and primary concern is trying to be relevant to sell courses. They don't care about you. Yes I used to listen to them (though not often), but now not anymore. The world doesn't get better or worse whether they choose to sip margarita on a beach or spin yet another "so many words to say absolutely nothing" video/blogs.
On bigger tasks OpenCode sometimes hangs. I'm not sure if that's on the provider side or on my network side, but i sometimes have to stop a subagent because its done but doesnt clean up. I wanted to try Pi, but now I'm wondering if Hermis is a better fit.
The Hermes subreddit is a curious place where every second person has quick their $300k a year job and is making a living off of Hermes doing... something? They treat themselves as the CEO of a bunch of agentic employees and have AI generated infographics of their "Stack" (all hail the mighty Stack) and everybody stands in a circle and applauds the most convoluted Hermes setups you've ever seen with not a single word how any of this is supposedly making anybody any money.
Are people developing like this? Are people asking money for something they one-shot instructions to an orchestration agent which delegated to fifteen other agents in Kanban and then spat out something that "works"? Every project I've ever had an LLM do a majority of the work for me has been strictly for my personal use; I'd never let anyone else use this stuff because it doesn't pass the vibe check. When a new frontier model comes out I'll pass the previous frontier model's work past the new model and let it tear it to shreds and see what improvements could be made for shits, giggles, and to waste a week's worth of tokens in the course of thirty minutes, but Reddit is overflowing with seeming non-coders who are passing this 100% organic slop off as sellable product?
I need fewer morals.