CTOs discuss controlling AI spending

AI costs force a reckoning for CTOs

Ameya Kanitkar, CTO at logistics software provider Larridin, stated that teams were unprepared for the scale of compute and storage costs once models moved from prototype to production. The issue often stemmed from “tokenmaxxing”—feeding ever-larger datasets into models to chase small accuracy gains.

Many organizations now admit they underestimated the infrastructure required to maintain these systems at scale. Some have even reversed earlier layoffs, rehiring engineers to manage the new complexity.

Bill Vass, CTO at consulting firm Booz Allen Hamilton, observed a similar trend. Three years ago, AI was a side project; today it consumes a large portion of his technology budget. The conversation has shifted from exploring possibilities to evaluating affordability.

Both leaders noted that hidden costs—data labeling, model fine-tuning, and energy consumption—often appear only after deployment. Vass explained that his firm now monitors these expenses in real time, a practice it skipped during initial experimentation.

Related: Key moments from the past week ahead

From pilot to profit: the new AI math

Panelists described a more cautious approach. Instead of pursuing every new model release, companies now focus on use cases with clear business value. Kanitkar revealed that Larridin scaled back some generative AI projects after discovering that simpler automation delivered most of the benefit at a fraction of the cost.

Budgeting has become more precise. Teams allocate funds by project rather than department and set strict limits on monthly cloud spending. Vass described a quarterly exercise where his team simulates a fictional company’s AI budget and tests it against scenarios like cost overruns or performance drops. The goal is to force leaders to consider trade-offs before problems arise.

This planning exposes a deeper conflict. AI projects often begin in isolated teams, where costs remain hidden. When those expenses surface later—usually after integration into workflows—they can create friction with other departments forced to cut budgets to accommodate them.

The change isn’t just about controlling costs. It also involves redefining success. Early AI advocates promised dramatic improvements, but many companies now accept modest gains—faster document processing, fewer errors, or small productivity boosts. The focus has shifted to whether a specific implementation justifies its ongoing expense.

As models grow more advanced, the challenges will intensify. The most sophisticated systems require specialized hardware, continuous retraining, and dedicated engineering teams. For most organizations, the era of inexpensive experimentation has ended.

One unexpected result of the cost crunch is stronger governance. Vass explained that Booz Allen now treats AI models like critical business systems, requiring documented performance metrics, failover plans, and clear ownership. This marks a significant change from earlier practices, when teams could deploy models without oversight.

Related: Chinese Tianwen-2 beams first image of Earth’s mini moon

Kanitkar added that Larridin now requires a “sunset review” for every AI project after six months. If a model fails to deliver measurable value by then, it’s discontinued. The policy has already terminated several projects that appeared promising but didn’t scale.

The financial strain has also revealed a skills shortage. Many companies hired data scientists to build models but neglected infrastructure teams needed for deployment and maintenance. These firms are now rushing to hire cloud engineers and DevOps specialists at high salaries to keep systems operational.

For CTOs, the challenge extends beyond technology. AI projects often begin with executive excitement, only to lose support when costs become visible. Vass noted that the most successful teams present AI as a long-term investment rather than a quick solution. This approach involves setting realistic expectations, securing early financial buy-in, and abandoning projects that don’t prove viable.

The executives agreed that AI isn’t a universal solution. Its value depends on how it’s applied. The companies that handle this period won’t be those that spent the most, but those that spent most effectively.