It’s frustrating to have all these 0days and CVEs to patch for now, but in the long run perhaps we’ll leave with more robust software.

It totally feels like the former if you watch it find and fix vulnerabilities or other debt-tasks one after another.

But only so many of these "technical debt" issue are little off-by-one errors that get remedied by a surgically narrow patch.

Many enough require some kind of restructuring or redevelopment of their own and it's going to take time to figure out if AI faces the same dilemma as human developers: where each non-trivial change carries the risk of some new issue, ad infinitum.

If that happens to be the case, and AI isn't able to magically produce perfect software in finite time, then what's really happened is that the threat of hostile AI exploiting our technical debt will force us to always be burning and churning code just to remain a moving target.

It's another "we'll see" thing for now, I think.

That is a choice I make every day when I get into an argument with Claude about the chosen design and how I find it insufficient. Does it slow me down? Sure, a little. But at least I know the foundation is bedrock instead of sand.

[0] https://www.bugsink.com/blog/http2shell-timeline/

Everybody in the industry knows that we've been building on quicksand. Moore's law meant that we could just sacrifice performance and add a dozen layer on top of technology from the 90s (if not older) and keep the thing chugging along.

Look at the PHP situation:

- HTTP is an old and quirky protocol with a lot of surface open for attackers that are rarely useful nowadays and might be better left off to higher layers. But of course, we can't break compat, so everybody still implements all the naughty parts, just in case.

- PHP was a joke language from the start, we were making fun of it in the early 2000s, the fact that over 20 years later it still powers a huge chunk of the web is enough to demonstrate that we're not serious people. Cue the PHP fans telling me that "it's good now". Maybe, I don't really care, the effort that went into making it good should have gone into ditching PHP for a better language 20 years ago. The bandaid mentality is what got us in this situation.

- And of course to add insult to injury, PHP, this security-critical language, is still implemented in garbo C using things like setjmp/longjmp. Just to make sure to absolutely maximize the damage if a vuln is discovered, I suppose. Brilliant.

- Oh and all of this is still running overwhelmingly on top of x86 which is its own bag of legacy garbage. "No but you don't get it, modern CPUs are really more like RISC CPUs, the RISC ISA is translated internally" yeah exactly that's my point let's stop doing things like that.

We're a clown industry. We only made it this far because, usually, buggy code doesn't kill people and we can iterate and patch things quickly. We never had to do things correctly, we could flourish with abysmal levels of quality acceptance. This may not be true anymore.

Of course the reason things are done this way is generally to maintain backward compatibility and make transition easier. At least now that we have LLMs this excuse will hopefully vanish: porting even huge codebases to a new language/framework, something that was a tremendous undertaking mere months ago, is now vastly easier and faster. We can leave the 1990s behind us.

One thing that I find confusing is you can always prompt a model to find more bugs, and invariably it seems to always be able to

Yes, AI developed code might have vulnerabilities or bugs, but so already has your JSP based web server that has Oracle and God knows what else among dependencies.

There is a concept of the hourglass model [0] of internet protocols, where IP sits at the waist. I believe that similar things are probably true in any "stack" based structure, such as computing, where the protocol at the waist has the most staying power (which at the current moment could be e.g. x86, or something) because it's essentially the universal translation layer that enables all the language protocols built above it to be agnostic to all the transport protocols below it. So, I see it as an open question as to what the best way to affect change in a situation like this is. I'm interested in if anyone else has any thoughts along these lines.

[0] https://en.wikipedia.org/wiki/Hourglass_model

AI removed 80k lines of code and nothing broke. much of this code was written by AI few months earlier.

All those "paying down" technical debt with AI, I would advise reading the code _very_ carefully. Sometimes the "solution" which passes some test or adheres to some described functionality is incredibly narrow / brittle, and/or has failure cases that are non-obvious.

Yes I'm using $latest_model on high, its still not there. Unless your threshold for "fixed" is "the 1M line lean proof runs, we don't know how it works", it still requires a lot of attention, hand-holding, and scrutiny, especially when fixing issues / working around failures (where it tends to shortcut or invent workarounds).

Thanks to The Lord everytime someone discovers the vulnerability he reports it right to the owners/maintainers and there are no bad actors out there keeping it as secret and using for years.

I'm not a PHP person, but so much of the world actively runs on it silently.

All technologies have gaps. PHP was affected differently not because it's old, but because it's so ubiquitous. Python is as old, so is Javascript, and many others.

So, the headline should read,

"AI is helping us eliminate 40 years of technical debt."

Since AI found the bug, and probably also generated the patch at this point.

An abundance of solutions is a problem for people who sell solutions.

i hope we get to a place where we keep the same number of developers employed as today, but leverage ai to make all software better across the board. hopefully the people in power realize that we've been understaffed this whole time.

Anthropic is arguing for formal verification [0]. This is an excellent idea on the edge of feasibility, but it's hard to see how this can work for the majority of software that has to adapt rapidly and are so open ended that it's difficult to even conceptualize all the input modes. It's an easy prediction that we will see verification on large trunks of fundamental software in the next few years, and a sub-verification confidence scheme for the software on top of it.

[0] https://www.anthropic.com/research/formal-verification

[1] https://www.theverge.com/ai-artificial-intelligence/949601/a...

[2] https://openai.com/index/coordinated-disclosure-of-critical-... - "To help more defenders use these capabilities, we’ve committed $1 billion in subsidized Daybreak access through Daybreak for Frontline Defenders, focused on organizations with limited security resources protecting essential services."

[3] https://www.anthropic.com/glasswing - "Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations."

My approach is https://tix.im/ - not launched yet. But from expedience: modeling everything as workflows of composeable nix flakes (expert systems from programs, skills, agents, workflows and their snippets, DBs blobs, service descriptions, ...) and then run everything in highly isolated environments. It is not only so much more efficient, reduces errors, increases quality, reduces unnecessary cognitive load, ...

I start to move more and more daily tasks into the systems. Even UI programs become interactive workflow steps, which allow me to attach from anywhere back to a task in an environment optimized for the task.

Dependency updates for example, a workflow. A subgraph from a security scanner expert checks each change in the dependency tree.

What do you think the chance of nonzero budget and time being allocated for this is, given that almost no company has seen real consequences to the business?

my webserver never needed a million lines of code just to pass bytes to other people

Just consider how laughable some working conditions in software are and then come to realize how architecture is shaped by the organization communication patterns and not reality. You will see it in a different light then.

AI is enabling us to fix longstanding software security issues that nobody had the time, skill, or desire to find

All a matter of perspective, I suppose.

Yeah, hmm a lot of people don't actually auto update their PHP so.. it's not like the vulnerability disappeared from the wild.

Aren't there still vulnerable services right now being hacked?

I fear it's only going to get worse.

Everybody wants a $1 Website, avoid security updates, pay nothing for maintenance, and done yesterday.

It has nothing to do with AI or a specific language or stack. As long as cost cutting will be the culture, driven by the economical model of competition, then we will accumulate technical debt BECAUSE it is the strategy.

... "by legacy sites"? :O

That's a pretty spectacular mischaracterisation of PHP's place on the web.

<cough> Wordpress <cough>....

AI does find things, but it is a small percentage the overall vulnerabilities.

IEEE is of course and industry owned organization that shills for BigAI, NSA and any other entity with money or influence.

This summer I had my Docker registry poisoned with malware artefacts, which again is something that no sane attacker would do before as it required downloading my images, carefully modifying them and then overwriting them, ensuring they work as before. Now it's just a simple prompt away. I am definitely ramping up my defenses, everything that doesn't need to be Internet connected goes behind double VPN and port knocking schemes, essentially making my infrastructure invisible from the outside, all unknown packet types get hard dropped, end-to-end encryption of all internal traffic, web endpoints behind CDN proxies and WAF providers (as much as I hate doing that), double and tripe security mechanisms everywhere, maximum restriction of privileges, ultrafast automated patching and AI based continuous monitoring, audits and interventions. You should also get used to that, this will surely still get much worse as attacker tooling gets better.

Well now, Great Responsibles, it's clear you too were out of your collective depth and were all paying for your smug hubris. Perhaps take a bite out of that humble pie you've obviously never tasted?

Instead we're spending the time churning out busy work fixing bugs that were too low value to care about a year ago. We have a new street lamp so we're working on anything it illuminates.

I haven't really seen the same kind of push towards verifiable software. It's mostly around solutions that will perpetually burn tokens and justify usage of the new toys.

Mine is a cynical take for sure but I honestly really miss meaningful feature work but that's not what the market's interested in right now, I guess

At my current day job, if I mention “hey, how about you update PHP to at least a supported version” people start whining about how that takes a lot of time and if really anyone would try to attack our public facing services written in… PHP.

Next time we get hit I can sit back, eat some popcorn and say “told ya’”.

Back then, you could log in as admin to any phpBB forum by copying a generic admin cookie from a shady web site. You could often then also compromise the hosting company that was running the forum, because nothing was really separated.

You would try to trick your friends into installing Sub7 so you could open their CD drive.

You could easily capture other people's data on public wifi networks in restaurants and hotels.

You could change the price of the pizza you were ordering through devtools in your browser.

You could infect half the world by emailing a picture of a blonde tennis player.

Sure, there have been quite a lot of pretty serious zero days recently. But many of them have been pretty hard to actually abuse, in part thanks to all these layers we've added over time. So I guess I'm not as pessimistic as many of you are. Especially because we all seem to be incredibly aware of what's happening and are acting on it.

I've been bullish on Rust for years because memory safety is simply too important and Rust can essentially prove an implementation correct and solve a certain class of very important bugs. It doesn't solve every problem and there is a complexity cost.

I've also said that writing multithreaded code is simply too complex. In an era of out-of-order execution, different levels of caching, asymmetric cores, hyperthreading (although this is largely dead now?), branch prediction and having to support multiple architectures it's simply too hard for most people to write correct multithreaded code. Period. If you find yourself as a non-library developer creating a mutex or spawning a thread, somebody has screwed up.

This is also why I think the single-threaded async/await model with post-request teardown of PHP is the correct choice for most people.

We're going to go through a dark period of a lot of 0days. As the article notes, this is paying 40 years of technical debt. It will result in a more secure ecosystem however. I believe this will ultimately also hamper black markets, governments and 3 letter agencies who trade in those 0days.

As painful as this all is, it will ultimately be a good thing, one of the few to come out of AI (IMHO).

Wow. Love it or hate it, but tell me you have no idea what PHP's place in the web ecosystem is without telling me you have no idea what PHP's place in the web ecosystem is.