Rising AI chip power — NVIDIA's H100 draws up to 700W, and rack density is climbing toward 50 kW — has pushed data centers past the limits of air cooling. Direct-to-chip and immersion liquid cooling now cut PUE toward 1.3 and water use per Microsoft's reported 0.30 L/kWh, while NVIDIA's GB200 NVL72 packages 72 Blackwell GPUs in a liquid-cooled rack rated at 25x the performance of H100 air-cooled infrastructure at the same power.
Why is rising AI chip power forcing data centers toward liquid cooling?
NVIDIA's H100 accelerator draws up to 700W in its SXM configuration and 350W in its PCIe configuration, a power level NVIDIA attributes to a TSMC 4N process and 80 billion transistors packed onto a single chipCITE:E1. Vertiv reports that as rack power density rises to 20 kW and quickly approaches 50 kW, traditional room-based air cooling is pushed to its capability limits for HPC infrastructureCITE:E2. Together, these two data points mark the point at which chip-level heat output and rack-level power density both exceed what air cooling was designed to handle.
What is the essential difference between direct-to-chip liquid cooling and immersion cooling?
Direct liquid cooling (DLC) circulates a dielectric coolant through cold plates mounted directly on chip surfaces, while immersion cooling submerges entire servers in coolant. Vertiv states that its direct-to-chip cold-plate approach is 3,000 times more effective at removing heat than air cooling alone for HPC infrastructureCITE:E3. Immersion cooling, by contrast, absorbs 100% of the heat generated by IT components because they are fully submerged in fluid, with single-tank systems delivering cooling capacities of up to 100 kW across 42U or 52U tank configurations, using either single-phase (non-boiling circulation) or two-phase (coolant boils at the chip surface and recondenses) designsCITE:E4.
How does liquid cooling improve energy and environmental metrics?
Liquid cooling is compressing both the power and water footprint of data centers relative to industry-wide averages. Uptime Institute's 2024 survey put the global average data center Power Usage Effectiveness (PUE) at 1.56, a figure it says has shown a continuing trend of inertia, while many recently built facilities — often specializing in denser IT architectures with direct liquid cooling — consistently achieve a PUE of 1.3 or betterCITE:E5. On water, Microsoft reports that a newer datacenter design using chip-level, closed-loop liquid cooling with zero operational water evaporation will avoid the need for more than 125 million liters of water per year per datacenter, and that its datacenters overall operated at an average Water Usage Effectiveness (WUE) of 0.30 L/kWh in its most recent fiscal year, a 39% improvement compared to the 0.49 L/kWh global average it reported in 2021CITE:E6.
| Metric | Prior / Baseline | Current / New Build |
|---|
| PUE (Uptime Institute, 2024) | 1.56 (industry average) | 1.3 or better (many recent builds) |
| WUE (Microsoft) | 0.49 L/kWh (2021) | 0.30 L/kWh (most recent fiscal year, -39%) |
| Water avoided (Microsoft, new design) | — | >125 million liters/year per datacenter |
| Rack cooling capacity (Vertiv immersion) | — | up to 100 kW per tank (42U/52U) |
How does NVIDIA's GB200 demonstrate the practical value of liquid-cooled servers?
NVIDIA's GB200 NVL72 is a rack-scale liquid-cooled platform integrating 72 Blackwell GPUs in a single rackCITE:E7. NVIDIA states that compared with H100 air-cooled infrastructure, the liquid-cooled GB200 NVL72 delivers 25 times more performance at the same power level, while also reducing water consumption and, according to NVIDIA, lowering the data center's carbon footprint and energy consumptionCITE:E7.
What this means
The chip-level power figures from NVIDIACITE:E1 and the rack-density ceiling reported by VertivCITE:E2 describe the same physical constraint from two angles — heat per chip and heat per rack — both landing liquid cooling as the response. Vertiv's own cooling-technology dataCITE:E3CITE:E4 and the facility-level metrics from Uptime InstituteCITE:E5 and MicrosoftCITE:E6 then show that constraint translating into measured outcomes: lower PUE at new builds and reduced water use at Microsoft's datacenters. NVIDIA's GB200 NVL72CITE:E7 packages both threads together in a single shipping product — a 72-GPU, liquid-cooled rack that NVIDIA positions against its own prior air-cooled generation on a same-power performance basis.