Nobody ever gets credit for fixing problems that never happened (2001) [pdf]
691 points - today at 12:38 AM
SourceComments
King Wen of Wei asked Bian Que:
“Of you three brothers, all physicians, who is the finest in the healing art?”
Bian Que replied:
“My eldest brother is the finest; my second brother comes next; I, Bian Que, am the least of the three.”
King Wen said:
“May I hear why?”
Bian Que answered:
“My eldest brother sees illness in the spirit, before it has taken shape, and removes it unseen; therefore his name is known only within our household.
My second brother treats illness when it is but a hair’s breadth from appearing; therefore his name does not travel beyond our village lane.
As for me, Bian Que: I pierce the blood vessels, administer strong medicines, and cut open the flesh. Thus, by such visible acts, my name has spread among the lords.”
Meanwhile, my perfectly purring department was struggling to keep the lights on.
It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing).
<insert IBM story about IT department cost cuts>
I'm not sure how we solve this, other than having management come from engineering.
My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course."
Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D
So I stopped doing all the administrative work and focused on just completing story points. A week or two later my manager asks the team "how come all of our meetings are falling apart now? We get into a meeting and no one knows what's going on."
I couldn't even get my own dad to pay for network support for his company since he would never pay my rate for anyone no matter what. After 2 other people failed to solve his problem I fixed it in 15 minutes and then he "really" didn't want to pay because it only took 15 minutes.
I was very good at what I did but got no appreciation for keeping things from breaking, only for fixing things after they broke. Marketing paid better, and I could point at real world numbers daily and justify my pay. I don't like it anywhere near as much, but at least it gets more respect than any other IT work I did.
Every place I've worked rewards the firefighter over the person who made sure nothing ever caught fire. And the worst part is the math is obvious to everyone except the people who set the incentives.
That illustrates the converse. People will absolutely try to avoid future problems if they are the ones that bear the consequences for them. Use birth control (or put your kid on it) so you don't have to raise another child for 20+ years. Don't hang out with that volatile "friend" who always seems to be having another crisis. Fix your roof so you don't lose the house. Don't go into debt because you're the one who will be paying interest on it.
But almost by definition, bearing the consequences of your own decisions implies that there's no transaction.
It's interesting and fitting that the article begins with a discussion of Toyota. People buy Toyotas because they want to avoid problems. The biggest selling point is reliability; a Toyota's value prop is that you can keep it for 20 years and you won't have unexpected things go wrong with it. Toyota has managed to turn this into a sales driver because they appeal to the self-interest of a buyer who will be living with the car for 10-20 years. There are thousands of individual decisions that Toyota makes to avoid problems with their cars, starting with a very conservative aversion to new technology or anything that is engineeringly risky. Each individual one is invisible to the customer, and often comes with significant costs in isolation. But because their sales driver is "be reliable at all costs" and they've ingrained that into the culture, they've built an organization that is willing to make these feature trade-offs for reliability.
Also, an interesting corollary is "Oftentimes, a good life is lived with few transactions."
- "Everything around is working just fine. What are we paying IT for?"
- "Everything is broken. What are we even paying IT for?"
Personally, I strive for the former rather than the latter; I like to say "If I do my job right, you never know I'm here." But that's what got me let go.
(and for karma's sake, I keep in touch with folks at the old company; it's an absolute crapshow. So I got that going for me; which is nice)
https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1012443 for the original, or https://realmensch.org/2017/08/25/the-parable-of-the-two-pro... for a reprint.
Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing.
Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.
https://news.ycombinator.com/item?id=8940820 - 24 Jan 2015, 50 comments
https://news.ycombinator.com/item?id=39472693 - 22 Feb 2024, 434 comments
But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years.
The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.
On other hand. for software engineering some of the signals that can be used to measure such a management itself can be
1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team
2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign
It read "This fixes a bug that hasn't happened yet".
It seemed really smart at first, but later I learned that the developer that added that code also had a pattern of appending spaces to the start and end of user input and comparing the length to 2 to determine whether the value was empty or not...
So I'm fairly sure "that hasn't happened yet" was probably more a case of "that I personally haven't introduced unnecessarily yet" :)
E.g. if the problems are quantifiable and there's a record, like dropping homicides from 100 per year to 20 per year in a city. Those extra homicides "didn't happen", but the improvement is understood.
For an one-off problem, it depends on how clear the path to the problem is. An electrician doing an inspection and noticing and fixing big electrical issues in the installation, would be appreciated, even if the accidents didn't happen.
No one notices when you cut 20% of some expensive process but cause no regressions.
Human groups work on shallow signalling and distributed confusion.
In other words, it’s not just a tool problem, any more than it’s a human resources problem or a leadership problem. Instead it is a systemic problem [...]
Shades of an LLMism, a bit padded, a quarter of a century ago. These days someone could easily give it a stink-eye. I'm sure that training has ingested this along with countless similar examples.
When everyone is technical to some degree, I find that credit for technical rescue is forthcoming.
Did you change from a quiet diligent one to manipulating and playing the game (now that you know the game)? Did you go from quiet and diligent to quiet and not diligent (why do good work when meh work does the trick)? Another path?
You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts?
1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and
2. They failed to ship and get subpar performance ratings for having no impact.
The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it.
You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own.
Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success.
Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything.
- Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it.
- Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19.
There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”.
Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash.
For some reason, these stories don’t make the news.
If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.
erhm, if this figure is close to true i can see what market ai companies is after.
Nobody ever gets credit for fixing problems that never happened (2001) [pdf] - https://news.ycombinator.com/item?id=39472693 - Feb 2024 (424 comments)
Nobody Ever Gets Credit for Fixing Problems That Never Happened (2001) [pdf] - https://news.ycombinator.com/item?id=8940820 - Jan 2015 (50 comments)