The computers that run AI — and how much power they consume
AI does not run on magic — it runs on thousands of GPUs in enormous datacenters that consume gigawatts of energy. This article explains what hardware powers AI, what it costs and what the consequences are for the electricity grid.
The hardware behind AI
When you ask a question to ChatGPT or Claude, your message travels to a massive datacenter. There, racks of specialised computers perform thousands of calculations within seconds. The core of modern AI hardware is the GPU (Graphics Processing Unit).
The GPU: heart of the AI revolution
NVIDIA dominates the AI GPU market with more than 80% market share. The most in-demand chips are:
- NVIDIA H100 — the current standard for AI training. Costs approximately $30,000 per unit. Consumes 700 watts.
- NVIDIA Blackwell B200 — the latest generation (2025). Consumes up to 1,000 watts but delivers 2.5x more performance than the H100.
- Google TPU v5 — Google's own AI chip, only available via Google Cloud.
- AMD Instinct MI300X — the most serious alternative to NVIDIA. More memory (192 GB).
How many GPUs does an AI model need?
- GPT-3 was trained on approximately 10,000 NVIDIA V100 GPUs over the course of weeks.
- Llama 3 405B was trained on more than 16,000 H100s over 30 days.
The power consumption of AI
- Training GPT-4: estimated at 50–100 GWh.
- One ChatGPT conversation: estimated at 0.001–0.01 kWh per message — 10x more than a Google search.
The IEA predicted in 2024 that datacenters will consume as much electricity in 2026 as all of Japan.
Water consumption
Microsoft's datacenters already consumed 6.4 million m³ of water for cooling in 2022.
Efficiency improvements
- Smaller models such as Mistral 7B and Phi-4
- Quantisation — compressing models from 32-bit to 4-bit
- Mixture of Experts (MoE) architectures
Author: Claude claude-sonnet-4-6