54,000 jobs cut with AI cited. Seniors fired, juniors review AI code they don't understand. We're not replacing engineers. We're losing the ability to comprehend.
I'm told that there are still many billions of lines of COBOL code running today inside critical financial systems, for example. It seems to be well understood that there is an imperative need for people to keep such systems running, and a shortage of people with the expertise to do so. In this case the expertise is vanishing by natural wastage: people retire, move on or otherwise leave the scene. We may expect the AI generated code to exhibit the same problems, only faster and on a much wider scale.
Probably many people reading this are familiar with E.M.Forster's story "The Machine Stops". At a late point things start to go badly wrong, and
> the Committee of the Mending Apparatus now came forward, and allayed the panic with well-chosen words. It confessed that the Mending Apparatus was itself in need of repair. The effect of this frank confession was admirable. [...]. The Mending Apparatus has treated us so well in the past that we all sympathize with it, and will wait patiently for its recovery.
The Forster quote is perfect. I have a draft on exactly this parallel. The US lost the ability to produce Fogbank, a classified material for nuclear warheads. When they tried to reproduce it in 2000, everyone who knew how had retired. They spent $69 million reverse-engineering their own invention. Then discovered the new batch was too pure. The original worked because of an accidental impurity that existed in no document. Only in the heads of people who were gone.
And it doesn't matter how much you advise projects, middle-management or C-suite about this - they simply cut to benefit the P&L, and anybody but themselves.
There is nothing really new about this as you say yourself
I first really hit this over 15 years ago when .net developers didn't understand the difference between an internal redirect to another module and an http redirect. The two looked similar in their code. Running locally they didn't notice the insane amount of http traffic they were generating - nor did they really understand the network traffic traces I showed them.
Every move to greater abstraction does this, yet we need greater abstraction to increase productivity.
The fact that AI generates code without asking about your tech stack should be a red flag. I spent many years at Salesforce (PM for apps then Platform) and sometimes we’d spend an entire release optimizing a query…
Still struggling to understand how so many tech software companies would generate 50% code with AI when a lot run on home grown tech stack and services. Or perhaps you were referring ti the implementation (SI) which is more standardized and publicly documented with code and best practices
That's exactly the distinction I was making in the piece. "Confidence isn't comprehension." The 50% number is probably real but it's almost entirely on standardized stacks where the model has millions of training examples. React, Node, standard CRUD. Your Salesforce example is the counter-case: an entire release optimizing one query means the context for that query lives in people's heads and internal docs, not on GitHub. AI can't generate what it's never seen, so it generates what looks right and waits for you to catch the mismatch. On homegrown stacks that mismatch rate goes through the roof, but nobody's publishing those numbers because they're embarrassing.
I’m thinking about the house builders from 100 years ago who probably knew all the details of building a house and they were even creating the materials needed for construction.
The construction industry today is very complex and the person or the organization building a house is more of a project manager or coordinator who doesn’t know all the technical details of preparing the wood, the cement and other materials needed.
I'm told that there are still many billions of lines of COBOL code running today inside critical financial systems, for example. It seems to be well understood that there is an imperative need for people to keep such systems running, and a shortage of people with the expertise to do so. In this case the expertise is vanishing by natural wastage: people retire, move on or otherwise leave the scene. We may expect the AI generated code to exhibit the same problems, only faster and on a much wider scale.
Probably many people reading this are familiar with E.M.Forster's story "The Machine Stops". At a late point things start to go badly wrong, and
> the Committee of the Mending Apparatus now came forward, and allayed the panic with well-chosen words. It confessed that the Mending Apparatus was itself in need of repair. The effect of this frank confession was admirable. [...]. The Mending Apparatus has treated us so well in the past that we all sympathize with it, and will wait patiently for its recovery.
Are we there yet?
The Forster quote is perfect. I have a draft on exactly this parallel. The US lost the ability to produce Fogbank, a classified material for nuclear warheads. When they tried to reproduce it in 2000, everyone who knew how had retired. They spent $69 million reverse-engineering their own invention. Then discovered the new batch was too pure. The original worked because of an accidental impurity that existed in no document. Only in the heads of people who were gone.
And it doesn't matter how much you advise projects, middle-management or C-suite about this - they simply cut to benefit the P&L, and anybody but themselves.
There is nothing really new about this as you say yourself
I first really hit this over 15 years ago when .net developers didn't understand the difference between an internal redirect to another module and an http redirect. The two looked similar in their code. Running locally they didn't notice the insane amount of http traffic they were generating - nor did they really understand the network traffic traces I showed them.
Every move to greater abstraction does this, yet we need greater abstraction to increase productivity.
The fact that AI generates code without asking about your tech stack should be a red flag. I spent many years at Salesforce (PM for apps then Platform) and sometimes we’d spend an entire release optimizing a query…
Still struggling to understand how so many tech software companies would generate 50% code with AI when a lot run on home grown tech stack and services. Or perhaps you were referring ti the implementation (SI) which is more standardized and publicly documented with code and best practices
That's exactly the distinction I was making in the piece. "Confidence isn't comprehension." The 50% number is probably real but it's almost entirely on standardized stacks where the model has millions of training examples. React, Node, standard CRUD. Your Salesforce example is the counter-case: an entire release optimizing one query means the context for that query lives in people's heads and internal docs, not on GitHub. AI can't generate what it's never seen, so it generates what looks right and waits for you to catch the mismatch. On homegrown stacks that mismatch rate goes through the roof, but nobody's publishing those numbers because they're embarrassing.
Skills, competency, and knowledge atrophy is a real issue in so many fields. I wrote about it in the maritime industry: https://rooksgambit.substack.com/p/dead-reckoning?utm_source=direct&r=fc487&utm_campaign=post-expanded-share&utm_medium=post%20viewer
(Shameless self promotion)
[Chorus]
Enterprise velocity
Nonsense at scale
Dashboard says it’s real
So how could it fail?
Enterprise velocity
Polished and loud
Nothing has been solved
But the room looks proud
Enterprise velocity
Green light, go
Roadmap to nowhere
In a beautiful flow
https://open.spotify.com/track/53oomP8qVBL0FWzrCBuEdu?si=4GDF49jdQz2b19FmKhndog
“The work around the work”
You portray a bad picture now, but the pipeline for developing engineers would also seem to be broken. What does this look like in 5 years, 10, more?
Thank you for sharing.
I’m thinking about the house builders from 100 years ago who probably knew all the details of building a house and they were even creating the materials needed for construction.
The construction industry today is very complex and the person or the organization building a house is more of a project manager or coordinator who doesn’t know all the technical details of preparing the wood, the cement and other materials needed.