liveBook · Manning

Energy consumption

As explained in Machines that Think

The amount of energy required by a system or process while it operates.

Energy consumption is the amount of energy a system or process uses while operating. For large language models, it is a constraint on both training and deployment: sustained computation requires power, while cooling, networking, and redundancy add overhead. section 4.6.1 section 4.6.3

Training relies on repeated numerical operations across accelerators such as GPUs and TPUs. Long runs can last weeks or months, so small inefficiencies accumulate. Networking bottlenecks, idle capacity, and poorly balanced workloads can leave accelerators underused while the system continues to consume resources. section 4.6.1

More efficient hardware, parallelization, and numerical precision can reduce the energy required per operation. Yet total consumption can still rise when models grow or usage increases. Efficiency per computation and energy consumption across an entire system are therefore different measures. section 4.6.3

Deployment choices also shift energy use:

  • Lower latency can require keeping models in high-performance memory rather than loading them on demand, increasing power draw. section 4.6.3
  • Higher throughput can favor large batches and centralized deployment, reducing per-request cost while concentrating energy use. section 4.6.3
  • Data center location, power sourcing, and cooling affect energy use beyond the model architecture itself. section 4.6.3

Power and scale

The human brain consumes roughly 20 watts while supporting perception, movement, reasoning, and learning. By contrast, training large AI models on conventional hardware can require megawatts across thousands of processors. These are comparisons of power draw in different systems and activities, not direct measures of equivalent work. Large training runs may consume gigawatt-hours over their duration, though estimates vary with architecture, hardware efficiency, and training strategy. section 5.3.4 section 4.6.1

Neuromorphic computing seeks some of the efficiency advantages of nervous systems through parallel, event-driven hardware. Its applications include robotics, sensing, and edge computing, where power is constrained. Such systems may simplify neural dynamics or limit learning mechanisms to make hardware practical, trading flexibility for efficiency and stability. section 5.3.4

How Machines that Think covers it

How deep it goes
Explains energy drivers and design trade-offs, using broad scale estimates rather than precise model-specific measurements.
Examples
Compares brain power draw with large-model training and estimates training-run energy in gigawatt-hours.
What you can do after
Evaluate energy trade-offs in training and deployment without mistaking per-operation efficiency for lower total consumption.
How it connects
Connects energy consumption to latency, throughput, data centers, and neuromorphic computing.

Also explained in

Compare how 7 books explain energy consumption →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.