Energy consumption is the amount of energy a system or process uses while operating. For large language models, it is a constraint on both training and deployment: sustained computation requires power, while cooling, networking, and redundancy add overhead. section 4.6.1 section 4.6.3
Training relies on repeated numerical operations across accelerators such as GPUs and TPUs. Long runs can last weeks or months, so small inefficiencies accumulate. Networking bottlenecks, idle capacity, and poorly balanced workloads can leave accelerators underused while the system continues to consume resources. section 4.6.1
More efficient hardware, parallelization, and numerical precision can reduce the energy required per operation. Yet total consumption can still rise when models grow or usage increases. Efficiency per computation and energy consumption across an entire system are therefore different measures. section 4.6.3
Deployment choices also shift energy use:
- Lower latency can require keeping models in high-performance memory rather than loading them on demand, increasing power draw. section 4.6.3
- Higher throughput can favor large batches and centralized deployment, reducing per-request cost while concentrating energy use. section 4.6.3
- Data center location, power sourcing, and cooling affect energy use beyond the model architecture itself. section 4.6.3
Power and scale
The human brain consumes roughly 20 watts while supporting perception, movement, reasoning, and learning. By contrast, training large AI models on conventional hardware can require megawatts across thousands of processors. These are comparisons of power draw in different systems and activities, not direct measures of equivalent work. Large training runs may consume gigawatt-hours over their duration, though estimates vary with architecture, hardware efficiency, and training strategy. section 5.3.4 section 4.6.1
Neuromorphic computing seeks some of the efficiency advantages of nervous systems through parallel, event-driven hardware. Its applications include robotics, sensing, and edge computing, where power is constrained. Such systems may simplify neural dynamics or limit learning mechanisms to make hardware practical, trading flexibility for efficiency and stability. section 5.3.4






