OpenAI, one of the world's largest customers of NVIDIA's graphics processing units (GPUs), has begun testing Google's Tensor Processing Units (TPUs) to power its AI systems, including ChatGPT. This move comes as the company faces mounting computational expenses and seeks more cost-effective solutions for its growing AI operations.
According to industry analysts, inference—the process where AI models use trained knowledge to make predictions or decisions—now consumes over 50% of OpenAI's compute budget. TPUs, particularly older generations, offer significantly lower cost-per-inference compared to NVIDIA GPUs, making them an attractive alternative despite potentially lacking the peak performance of newer NVIDIA chips.
"While older TPUs lack the peak performance of newer Nvidia chips, their dedicated architecture minimizes energy waste and idle resources, making them more cost-effective at scale," explained Charlie Dai, VP and principal analyst at Forrester. Industry analysis suggests Google may obtain AI compute power at roughly 20% of the cost incurred by those purchasing high-end NVIDIA GPUs, implying a 4-6x cost efficiency advantage.
However, OpenAI has clarified it has no immediate plans for large-scale TPU deployment. A spokesperson told Reuters the company is in "early testing with some of Google's TPUs" but currently has "no plans to deploy them at scale." This cautious approach reflects the significant technical challenges involved in transitioning infrastructure, as OpenAI's software stack has been optimized primarily for GPUs.
Beyond cost considerations, this move represents a strategic diversification of OpenAI's compute sources beyond Microsoft, which had served as its exclusive data center infrastructure provider until January 2025. The company has already partnered with Oracle and CoreWeave on its Stargate infrastructure program and is developing its own custom AI processor, expected to reach the tape-out milestone later this year.
The implications for the AI hardware market could be significant. If successful, OpenAI's adoption of TPUs could validate Google's hardware as a viable alternative to NVIDIA's near-monopoly in high-performance AI computing. This could pressure NVIDIA to innovate or adjust pricing while creating new competitive dynamics between cloud providers like Google, Microsoft, and Amazon as they vie for dominance in AI infrastructure.