A natural evolution of engineers losing touch with the customers and users.

I'm noticing some of the concern play out regarding AI weakening the capabilities of software people.

I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patient, old school troubleshooting.

I think what's happening is the AI system draws poorly aligned and led engineers into this ego inflation feedback loop where they are completely detached from reality because these tools can simulate a better one.

Anyone who's worked in tech in a large company will probably have experienced having an ops team who use RPA tools to do repetitive tasks that tech teams get the blame for when things break. AI will make this so much worse. Things will break, everyone will assume 'tech knows the system', but really it's a new process outside of the tech teams that someone vibe coded but got it wrong.

Audit trails, logs, and tight data governance where things can only be accessed with proper roles is the only possible solution.

If an RPA team ever gets direct access to a production database in your company, look for a new job.

We have been running Agents on infrastructure and letting to create resources, scale up and down, security scans etc.

I agree with premise of thr blog. The question i have been asking internal does knowing your system really matter if you can recreate it in minutes.

We recently had a situation, where in with our internal platform and claude we recreated everything in minutes.

Management in the end cares about the outcome and not how the meat is made.

If capability increase continues as it has, then an incident that cannot be resolved by AI will stump humans no matter the practice.

I like the plane example from the article,but I think in reality it will be like code. 1.5 years ago engineers would routinely say that they still write code by hand here or there to keep their skills sharp, and that's just not something you hear much if at all.

If an SRE is faced with a situation an AI can't solve, then said SRE will use the AI systems to triage further, point it to different places and so on.

This works for SREs with pre-AI experience and intuition, possibly less so with new recruits coming in post-AI. I don't know what the solution to this is, maybe practice drills is it, but I have a hunch the entire field will be subsumed, same as many other engineering fields.

There is only so much need for taste and judgement, before even that has been incorporated into the models.

The more code writes autonomously, the less intuition the human owners have about that code. Loss of intuition is a seed of technical debt that grows with time. Over a long enough horizon, it can make looking at your own codebase feel like the first day on the job (sometimes at a company you started).

Luckily, there are ways to mitigate this and essentially translate those human intuition of how the codebase “should” be into guardrails for the agents. But without that, your setting your sails in a stochastic sea where each wave looks nothing like the last.

Isn't there anywhere to "go" from here? In the last decades, introducing new high level abstractions on top of existing paradigms naturally had everyone move up the ladder and work at the next higher level, why should this be different these days? Do we think AI will reach the top of the abstraction ceiling, so there's no where to go from here?

I find the use of AI like quicksand.

The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed.

Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.

Good article and I like the callouts to the aviation industry. For me what's missing is the author should also have touched on CRM and SRM.

Also, that paper "The Ironies of Automation" is one that everyone should read. It's fairly short.

There is a related problem in terms of these situations where the computer system is handing off to the human. It's called "the bumpy transfer of control." Very fascinating concept.

Ironies of Automation is front and center in this article which is awesome. Many of the conversations on AI automation are describing or rediscovering the insights the paper covered.

"THIS IS NOT A DRILL"