Extraordinary claims
AI and the benefit of the doubt
Every year, in every organisation, budget season rolls around. The big contract renewals need a business case, and so does any new spend. A disciplined company looks for problem-solution fit and projected returns. The right questions get asked. What problem does this solve, for whom, and how will we know it worked?
These investments are pressure-tested as the company’s being asked to make a bet that they will return more value than they cost. The review is not bureaucracy. It’s a sensible ritual when done well. Plenty of organisations do this badly. Renewals roll over on autopilot, and nobody checks whether the saving ever arrived.
That ordinary, everyday slackness is nothing next to what we are seeing now.
Carl Sagan popularised the phrase that extraordinary claims require extraordinary evidence. Generative AI has proved the opposite. The claims made for it by its vendors were so large that no evidence was deemed necessary.
When the promise is that a technology will out-think us, write our software, and stand in for whole functions, the usual tests stop applying. Nobody runs a ten per cent rate-of-return calculation against a singularity. As the conversation about AI strategy moved out of the back corridors of engineering and into the boardroom, so the pressure to invest heavily in AI came.
This is not a failure of knowledge. Nobody in the room has forgotten how to write a business case. They have decided that this is not the moment to insist on one.
Fear of being left behind features frequently in the rhetoric around why companies invest. When every competitor is announcing that they are now an AI company, it takes a brave chief executive to tell her board she is not convinced. The cost of being wrong about a revolution feels existential; the cost of overspending on a revelation is a line in next year’s accounts. The benefit is assumed, the evidence is a formality we’d attend to later, and the only real question on the table is how much.
In 1841, Charles Mackay catalogued what happens when a claim gets big enough. Extraordinary Popular Delusions and the Madness of Crowds tracks schemes that swept up careful people who could each give you a careful reason for joining. Through tulip mania and the South Sea bubble, the shape is always the same. The claim is large enough that standing aside looks like an irrational act, and the cost of being the lone sceptic who turns out to be wrong is unbearable.
But signs are emerging that the crowd is getting restive.
As subsidised subscriptions are replaced with token-based cost models, cost discipline is back on the agenda. While productivity gains are being documented, the transformative impacts of AI on the economy haven’t arrived yet and the economics fuelling the AI vendors have come under scrutiny. The valuations have floated free of the revenues meant to justify them. Ed Zitron, who has built a following warning that the arithmetic doesn’t add up, sets combined AI revenue in the tens of billions against capital expenditure in the hundreds. Man Group reports that ”AI technology is transformative and here to stay. But the inflated financial architecture supporting it may be unsustainable.” Mainstream news sites carry articles anticipating the end of the AI bubble.
Deloitte has found that ROI on generative AI investments takes longer than traditional technology investments in part because AI rarely delivers value in isolation. It is typically introduced alongside efforts to improve data quality, reconfigure teams or streamline operations. This opens the question of how much return is the result of AI, and how much in updating historically inefficient practices. Separately, an MIT study found that 95% of generative AI projects fail because organisations aren’t doing enough to modernise their processes to support the investment.
The thing is, the usefulness of generative AI does not depend on the survival of the companies selling it. Open models lag only slightly behind closed models, having all but closed the gap in 2024. These models run on a company’s own hardware, against its own data, rather than being metered by the token. If the labs that burned the capital go under, the majority of the capability survives them.
The tension of rising costs and limited successful applications means that AI investments are now being subjected to the same tests we apply to everything else. Deloitte has seen organisations prioritising ROI and focusing their investments on high-confidence initiatives.
If the vendor valuation bursts, the technology will still be available. The companies that thrive will be the ones with something to show for the spend, the ones still asking what problem this solves, for whom, and how we will know it worked, while everyone around them buys on faith.
Extraordinary claims, once they get extraordinary enough, stop being asked for any evidence at all. That is precisely the moment to keep asking.