Amazon Web Services (AWS) has introduced a groundbreaking cooling solution to tackle one of AI's most pressing infrastructure challenges: managing the immense heat generated by today's powerful AI processors.
The In-Row Heat Exchanger (IRHX) represents AWS's custom approach to cooling Nvidia's latest generation of graphics processing units (GPUs), which are essential for training and running large AI models but consume enormous amounts of energy. Traditional air cooling systems, sufficient for previous GPU generations, simply cannot handle the thermal demands of Nvidia's Blackwell-based chips.
"They would take up too much data center floor space or increase water usage substantially," explained Dave Brown, Vice President of Compute and Machine Learning Services at AWS, regarding why existing cooling solutions were inadequate. The IRHX addresses these limitations by circulating chilled liquid close to server rows through a system of pumping units, water distribution cabinets, and fan coils, effectively removing heat from densely packed GPUs without requiring wholesale data center redesigns.
The cooling technology works in tandem with AWS's newly launched P6e instances, which leverage Nvidia's GB200 NVL72 platform—a supercomputing system containing 72 Blackwell GPUs interconnected within a single rack. This configuration delivers 360 petaflops of computing power and over 13TB of memory, enabling customers to train much larger AI models more quickly and efficiently.
Beyond performance benefits, the IRHX employs a "closed loop" cooling mechanism that recirculates coolant without increasing water consumption, making it environmentally sustainable. AWS estimates its newer cooling systems can reduce mechanical energy consumption by up to 46% during peak cooling conditions compared to previous designs.
The development of IRHX aligns with AWS's broader strategy of developing custom infrastructure components. In recent years, the company has invested heavily in creating its own chips, storage systems, and networking gear. This approach has proven financially beneficial—in the first quarter of 2025, AWS achieved its highest operating margin since the unit's creation.
While other tech giants like Microsoft, Google, and Meta are pursuing similar custom hardware strategies for AI workloads, AWS maintains a competitive advantage through its extensive global data center footprint and experience deploying custom hardware at scale. The IRHX technology further strengthens AWS's position as a leader in AI infrastructure, making advanced AI computing more efficient, sustainable, and accessible.