When AI costs more than it saves
Enterprises are paying too much for AI
Following up last week’s essay about Uber blowing its AI budget and Microsoft scaling back its Claude Code spend, Sam Altman has stated that AI budgets are becoming an increasing concern for their customers.
If Open AI is involved, that means the primary budget in question is a token budget. Users, mainly developers, using tokens to build their products.
Teammate AI automates business processes. Our automations have starts and finishes.
We do, basically, the same thing every time. Take reading a PDF invoice.
The data on the invoice differs (thereby requiring that we use an AI model to read it), but reading the invoice, structuring the data, and doing something with that data is the same, otherwise.
Since we know what “success” looks like, both we and our clients know before we start the costs. Surprises are rare.
Making up numbers, we know that the AI cost of reading each page of an invoice is $0.05. There’s no confusion. If we sell that for $1.00/page, we make money.
When it comes to software development, with token-based billing, the costs can become unlimited.
Imagine your use of Claude or ChatGPT or Gemini on a daily basis. If that were token-based billing, you’d be very hesitant to use it frequently. The AI rarely outputs exactly what you want the first time you ask it a question or to do a task. So you refine it throughout the conversation and, over time, you achieve the output your desire.
If you needed to pay $5.00 per 10,000 words (input and output), you’d feel kind of ripped off every time the AI did or said something stupid.
That’s what’s happening to these enterprises with software development. AI might be generating the majority of the code at these firms, but a significant percentage of that AI-generated code are bugs. Then, the AI is sent out to fix the bugs. In the process, it may fix them, it may not.
It’s faster, don’t get me wrong, but it’s expensive. Uber has set a $1,500 / developer / month cap for AI. And, there are only so many new features that most companies can build in a year before the rest of the organization runs out of capacity to absorb them.
So what happens next?
Not sure, really.
They could increase AI budget and decrease the number of developers, but that can become a real issue if AI token prices rise.
They could reduce AI budget and start using more human developers, but the pace of development will decrease.
My sense is that some companies will chose option A and some will choose option B, and we’ll find out which works. In the meantime, I wouldn’t be surprised if we see wavering in the valuations of Anthropic and Open AI.