AI factories are characterised by soaring power densities and sharp swings in IT load
Two trends make AI factories different from traditional data centres. First, they are becoming
increasingly power-dense, meaning they have higher power demand per unit of space.
Second, they tend to house much more variable electrical loads. The first trend is driven by
the growing power rating of chips (i.e. GPUs), as well as by the clustering of more chips into
a single rack.4 Multiple chips are networked tightly together to function effectively as a single
computer. This trend is driven by the need to increase computing power and memory while
minimising both electrical losses and data latency. The power requirements of the associated
auxiliary equipment must also increase to maintain optimal conditions for the higher power
density of the IT equipment. For instance, since more heat is generated at the rack level,
cooling equipment must scale accordingly.
For example, Nvidia’s Ampere architecture from 2020 had a rated power of around 400 W
per chip and clustered 32 chips per rack, resulting in a power density of around 13 kW per
rack, including central processing unit, networking and storage power. The current Blackwell
architecture has a rated power of 1 000 W per chip and 72 chips per rack, with a power
density of 130 kW per rack. The announced Rubin architecture will reach a power density of
600 kW per rack and will pave the way for 1 MW per rack density. To put this in perspective,
with the announced Rubin architecture, a box the size of a household fridge would have a
peak power draw equivalent to that of around 65 households. This rate of increase in power
density has no historical precedent in electrical engineering.
"To put this in perspective, with the announced Rubin architecture, a box the size of a household fridge would have a peak power draw equivalent to that of around 65 households. This rate of increase in power density has no historical precedent in electrical engineering"
