Liquid cooling in the data centre

Last month, I gave a private briefing to some investors about liquid cooling in the data centre. What's the road ahead for this technology?

For those curious about how liquid cooling works, I've attached a small section of the deck showing systems used in data centres today.

Understanding a data centre isn't difficult, though it sure can get confusing. Here's what I talked about.

Why liquid cooling, why now

Liquid cooling has been around for quite a while now. Why the sudden interest? One word: AI.

• Scaling laws suggested better AI with more GPUs.
• Justified ever-larger GPU deployments.
• No result. Then one day, AI worked.

The result?

Massive scale, powerful AI systems that must be cooled. And a liquid such as water can carry about 3,500 times more heat than the same volume of air. So liquid cooling it is.

Types of liquid cooling

The 3 key types of liquid cooling are:

• Rear Door Cooler (RDC) - Chiller water to each rack.
• Direct to chip (DTC) - Chilled water to CPUs/GPUs.
• Immersion cooling - Immersion in a liquid (not water).

You can check out my photos of the various liquid cooling solutions in the attached slides.

Key components

I highlighted some key components, and how they typically fit into a data centre.

• Cooling Tower.
• Chiller.
• CRAC.
• CDU:
  (a) Liquid to liquid CDUs (Typically).
  (b) Liquid to air CDUs.
• Manifold.

Additional topics

I also talked about the following topics, though I capped it at just three of each due to time limitation.

(a) Myths and realities
(b) Challenges for data centre operators

What's next

Looking at data centres today, I do think two things will happen in the months and years ahead.

• AI data centres will diverge even further from those in traditional data centres. As noted at my fireside chat two weeks ago, this is already happening today with new, high-capacity systems designed for AI workloads.
• Inference hardware will start to increase. Some desktop and smartphone can already do some inferencing, but I will expect more inference-capable AI server chips to proliferate. Did you know: The China-only H20 is much stronger in AI inferencing than AI training?

Where do you see the data centre industry going?

PS: And yes, my presentations tend to be very graphical with few words.