AI-powered. Or AI-powdered?
The difference is visible in your org right now. Your customers already see it.
TL;DR
The article makes five claims that matter.
Claim 1: What looks like AI winter is a trust collapse - in code, in agents, in orgs that deployed AI before building the capacity to govern it.
Claim 2: Meta ran the enterprise-scale experiment. Two AI security failures in six weeks were the logical output of gutting infra and security capacities to feed model training.
Claim 3: Verification debt is now measurable. 42% of committed code is AI-generated. Half ships unverified.
Claim 4: The root cause is absent measurement architecture. Organisations deploy AI to hit a deployment metric and never scope the verification layer.
Claim 5: Three simple tests tell you where your org actually stands - on outcome gap, consequence accountability, and verification cost. None require a consultant.
I started last Friday trying to get a package sent to the right address and a 22-euro train refund.
I ended it thinking about what it means to build something that genuinely serves people versus something that processes them while appearing to.
The question kept buzzing in my head for a few days.
The AI deployment - will it work? It probably will.
But if it half-works, who carries the cost?
Let me start from the beginning.
Last Friday I called local French telecom operator - Bouygues.
They had shipped a packaged to an address I gave them 5 years ago. My current address is in their system. My fiber optic installation, phone lines, previous deliveries - all registered here. Still. The package went somewhere else.
I call them. A voice bot asks me for a reason for my call. I know how these go, so I skip the dancing around: “I want to talk to a human".
“Sure,” says the bot. “But what’s the reason?”
“Let me talk to a human”.
“Sorry, can’t help you”. It sends me an SMS with a link to a support chatbot app. And hangs up on me.
Surprised, I call again. The bot recognizes me. “Is this the same problem you called before for?”
Sure it is. I say yes.
“Sorry, can’t help you”. Another SMS. Another hang-up.
Frustrated, I make a third call. I say no, different problem, making something up. I get through to a human operator. She confirms that they already heard from angry customers about the voice bot. The address problem is fixed instantly.
The problem solved. The bot is still there. It still “thinks” I have a different problem.
Later that day I try to claim a delayed train ticket refund from SNCF - local railroad company. They have a rule - G30 - guaranteeing partial refund if you train arrives more than 30 mins late. Mine was 1h40 late. As my bookmarked link didn't work, I find the new form page.
The form page tells me I cannot fill in a form. I must go through their chatbot TOUTOUI (pronounced in French like ‘tutuy’)
Fine, I log in. TOUTOUI asks me: “How many passengers? More than 10 or fewer than 10?”
My order is in my profile, the session is open. My passenger count is one line of data it already has.
I answer the question - LESS THAN 10. It finds my trip, acknowledges that the train was indeed delayed. Then sends me an email saying that they would start calculating the refund. That I can follow up on my claim. Through TOUTOUI. The bot knows the delay, it was confirmed as 1h40. The bot knows the range of delays. It knows what percentage must be refunded. Yet I have to follow up on my claim via the chatbot.
Two encounters. One Friday. Both solved, technically. Both leaving what I can only call an aftertaste of being AI-Powdered.
Yes, not AI-powered. AI-powdered. Sprinkled on top.
The appearance of capability without User Experience architecture to back it up.
The thing about powdered AI is that it looks fine until you need it
Bouygues Telecom’s AI wasn’t bad. But it failed because they deployed it to eliminate the human part without first ensuring the AI path actually worked. The bot’s job was to reduce the call volume. It did - by hanging up on customers. Metric achieved.
SNCF’s chatbot was almost competent. But it still failed because someone optimised for chatbot usage rate. And built a system that forces every customer to go through TOUTOUI regardless of whether it adds value. Engagement metric achieved. Customer experience: powdered.
I think this pattern is worth saying out loud.
Not “AI doesn’t work”. Not “AI winter is coming”. These are wrong statements and they produce the wrong reaction - either panic-freezing everything, or doubling down on the same cosmetic deployments with better PR.
The right question is:
Did your organisation build AI that SERVES users? Or AI that PROCESSES users while appearing to serve them?
The distance between these two things is a governance gap.
Meta ran a larger version of the same experiment
Bouygues and SNCF are customer-facing symptoms. Meta shows the organisational disease at scale.
In April 2026, Meta cut roughly 8,000 people - around 10% of total headcount - and reassigned thousands of senior engineers from infrastructure, security and core product work into a new AI tooling and data-labelling organisation.
The same week, keystroke and mouse-movement tracking was deployed across the company to capture training data, triggering enough internal backlash that leadership had to add pause and exemption rules.
The logic is understandable. Apply the same method Meta built for 20 years - track user actions across Facebook and Instagram, etc - to train better internal models.
Then a Sev-1 security incident in March 2026. An internal AI agent gave a flawed recommendation to an engineer. The engineer followed it. Sensitive data exposed internally for two hours. Meta tightened internal review protocols.
They learned the lesson. Applied it internally. Moved on.
The customer-facing AI support bot kept running.
Between May 31 and June 1, attackers used that bot to hijack Instagram accounts. No technical skill required. Open the official support assistant. Ask it to add a new email to an account. The bot complied - the identity verification check that should have stopped this was broken elsewhere in the code, and the bot never knew.
Named victims: the Obama White House Instagram account, defaced with pro-Iranian messaging. Sephora. The Chief Master Sergeant of the US Space Force. At least 20,225 accounts compromised per Meta’s own filing. The New York Times later put the number at 34,000.
Meta’s VP of Communications: “This issue has been resolved.” Their CISO resigned (per internal reporting).
After the internal incident, they tightened internal review. The customer-facing bot - running the same class of logic, facing 20,000 users instead of one engineer - was not in scope.
That is AI-powdered at enterprise scale.
The verification debt
Sonar’s January 2026 survey of 1,100 developers shows that AI now generates 42% of all committed code. 96% of those developers do not fully trust what the AI wrote. Only 48% always check it before it ships.
So roughly half the AI-generated code going into production right now is unverified by the person who committed it.
Lightrun's 2026 engineering report goes one layer deeper. 43% of AI-generated code changes require manual debugging in production - after passing QA, after passing staging. Zero engineering leaders in that study expressed confidence in AI code behaviour after deployment. The average team needs two to three redeploy cycles to verify a single AI-suggested fix.
The GetDX longitudinal study tracked 400 companies over 15 months. AI tool usage increased 65%. Pull request throughput improved by roughly 10%. The authors’ conclusion:
Coding was never the bottleneck.
Review, planning and handoffs - the human judgment layer - remained almost entirely untouched by the AI investment.
Almost right, but not quite.
That’s TOUTOUI asking for your passenger count when your order is already open. That’s the Bouygues bot recognising you and hanging up anyway. Scaled into every codebase where the AI wrote the code and the developer committed it without looking.
Let’s steelman all this. You can argue that Meta is a hyperscaler. Telecom and railroad companies are French players with legacy IT debt. None of this applies to your org. Right?
Maybe.
But the Sonar and GetDX data does not come from hyperscalers. And the verification layer was never scoped at Bouygues either.
Scale changes the consequences. It does not change the decision that caused them.
This is a verification debt:
An invisible cost accumulating inside every engineering team that deployed AI tooling without deploying the judgment.
The consequences arrive as a high severity security incident. As a customer hanging up for the third time. As 34,000 Instagram accounts handed to attackers by a bot that was functioning exactly as designed.
The verification layer was never built. I bet, it was not even scoped.
In every transformation failure I have been close to, the pattern was the same. Decisions made on political momentum and opinions, not on data. The technology was secondary. The measurement architecture - or its actual absence - was what decided the outcome.
AI deployments are repeating it at speed. Bouygues measures call deflection. SNCF measures chatbot engagement. Meta measures token throughput. Nobody measures whether the customer got what they called for. Nobody measures what the bot did to the account it was supposed to protect.
So here are tests. Run them on your own org before similar failures happen.
The simple three tests
Test 1 - the outcome gap
Pull your AI customer-facing deployment’s primary metric.
Whatever it is - call deflection rate, chatbot sessions, ticket closure rate.
Now find the customer outcome metric it was supposed to improve: resolution rate, repeat contact rate, customer effort score, NPS on that channel.If you cannot find the outcome metric or it is not being tracked, you are measuring deployment success, not customer value.
That is AI-powdered by definition.
The threshold: if your org cannot produce both numbers in the same meeting by September 2026, the deployment was built to justify itself, not to serve anyone.
Test 2 - the consequence inventory
List every AI agent or AI-assisted workflow in your production environment that can take a consequential action without human sign-off.
Consequential means: changes a customer record, modifies access permissions, routes a transaction, alters a configuration, sends a communication on behalf of a user.For each one: who reviews its outputs, at what frequency, and who is named as accountable if it fails?
If your answer to the last question is a committee, a framework document or noone - you have the same architecture Meta had in May 2026.
The threshold: if more than one such workflow exists with no systematic review owner by September 2026, the incident has not happened yet. But it is already scheduled.
Test 3 - the verification cost
Ask your engineering leads one question this week: what percentage of their team’s time goes into reviewing, fixing and validating AI-generated output?
The threshold: if your engineering org cannot answer this question with data by September 2026, you are accumulating verification debt.
Three questions. None of them require a consultant. All of them have been answerable since the day you deployed AI tech in your org.
As usual, my final ask this week
These Signals come from conversations with executives who already face these decisions. If this one named something you have been circling - you know who in your org needs to read it before they commit.
And if nobody comes to mind, that is probably the answer to Test 2.
Share this Signal
And if you want to support the work directly:
Good piece, and it lands the simpler problem plainly: NLP isn't new, and what most organisations learned early on was that you need someone interpreting what a person is actually asking for before you can address it. That interpretation layer hasn't gone away just because the bot got more capable — it still needs building in, and I've sat on calls where the technology is the thing making me angrier, not the problem I called about.
There's a sharper issue underneath that, though. Someone with a cognitive impairment, or in the middle of a mental health crisis, needs help at the point they're asking for it — not a bot that can't recognise the state they're in and routes them into another loop. That's not a general design flaw; it's a question of who an organisation is deploying a given technology to, and who it quietly can't support. One size doesn't fit anyone in a crisis.
That's the gap under your Test 1. The outcome metric gets chosen by the organisation, but the population who can't be served by a given deployment is rarely part of that choice — the vulnerable user the bot fails is invisible to a metric built around deflection or engagement, by design, not by accident.
I had one of the best experiences of this kind of technology years ago with an early Apple assistant — say one sentence, it understood you, it did the thing. Simple. Now the same category of tool is more complicated and works about half the time, and every call ends the same way: just put me through to a person.
Tout oui - j’ai bien rigolé 😂
You’re pointing at a new “disease”. Everyone wants to ship fast because AI is magic, right?
Nothing changed, with or without AI. You have to write acceptance tests, run automation, functional tests, etc.
My experience building my business on AI is that you need to go slow (1) and you should only automate stable and well-documented processes. Normally that automation should be there in Enterprise already and you’re just adding the AI layer.