AI's Brutally Concentrated Economics: 3% of Investments Generate 60% of Returns

You can’t blame people who believe a massive AI investment bubble is at risk of bursting, triggering a crisis on the scale of the dot-com bust or worse. There are increasingly urgent concerns about massive capital spending, soaring valuations, high debt levels and the circular nature of AI firms pouring money into other AI firms.

And sure, a handful of players would probably escape a sector collapse and go on to change the world, just like the dot-com survivors did. But even the most likely eventual winners in AI are losing billions of dollars right now.

It’s hard to predict how this will play out in the financial markets, but here’s a clue. Keep an eye on demand for AI, measured in units of data processed. It’s soaring right now. The entire AI bet may turn on how far and fast it ramps from here. 

A key mistake occurred a year ago when AI leaders focused solely on AI cost curves for a unit of computing and took their eye off the number of units needed to get the work done, said Heath Terry, the global sector lead for technology and communications research at financial services firm Citi. AI computing costs had been declining around 90% every seven months, a dynamic akin to Moore’s Law for microchips, giving AI companies reason to believe their price-performance ratio will improve.

Then usage, which everybody expected to increase as the price fell, went absolutely parabolic.

“This time last year, we were processing 9.7 trillion tokens a month across our products and APIs [application programming interfaces],” Alphabet’s Google said in a May blog post, referring to units of AI usage. “Now, we’re processing over 480 trillion—50 times more.” 

Google was just getting warmed up. “Since then we have doubled that number, now processing over 980 trillion monthly tokens, a remarkable increase,” the tech giant said in July during its second-quarter earnings call. 

This month, Google said the figure had reached 1.3 quadrillion.

(News Corp, owner of The Wall Street Journal, has a commercial agreement to supply content on Google platforms.)

More reliable answers, at a price

The demand for tokens has been driven in part by the fight against the “hallucinations,” or incorrect or nonsensical outputs stated as if they are true, notoriously produced by generative AI models. Current cutting-edge models are designed to counter the problem often with the help of a kind of reinforcement learning in which they answer the same question multiple times internally before issuing a response. They also might query a so-called mixture of experts, meaning specialized models. That has helped AI produce better answers—and exacerbated AI’s resource needs. So the unit price of a token has fallen, yes, but the overall consumption of tokens has gone nuts.

That demand will only continue to soar. OpenAI’s new Sora short-form AI video app powered by the latest version of its Sora video generator is a hit. (News Corp has a content-licensing partnership with OpenAI.) The company is looking to ramp up efforts in AI commerce as well. And Anthropic’s new large language model, Claude Sonnet 4.5, can code for 30 hours straight. Such longer-thinking models are an emerging form of AI that will put the industry’s ballooning data-center capacity to use.

AI startups are losing tens of billions of dollars more than they would have partly because of this dynamic. Startups and venture-capital investors so far are underwriting such losses, much as Uber Technologies once subsidized ride prices for users to build a market, Terry said.

Better economics, maybe

Three forces are on track to come together and push the economics of AI into the black, in Terry’s view.

First, models will continue to improve. While the evolution of the transformer architecture underlying generative AI such as large language models may have slowed down, developers are creating new forms of hybrid models such as neurosymbolic AI. Some AI developers say they are closer to the point where AI systems are beginning to assist in their own improvement, a concept that Terry calls AI squared.

“Over the last few months, we’ve begun to see glimpses of our AI systems improving themselves,” Meta Platforms Chief Executive Mark Zuckerberg said in July on the social-media giant’s second-quarter earnings call.

All of that should make AI models more efficient, which could have a beneficial effect on the economics of AI.

Second, data centers under construction around the world will soon begin operations, increasing the supply of computing power and energy and driving down the cost of training and using AI. Tech company Microsoft last month said it is in the final stages of construction on a $3.3 billion data center in Wisconsin and announced plans for a second, $4 billion facility in the area.

And third, increasing demand for tokens could finally become an economic benefit as models improve and the supply of computing infrastructure expands.