A response from a large language model (LLM) may appear instantly, but the system behind it uses electricity. Energy consumption is the electricity used to train a model or generate responses with it. Inference means using a trained model to produce a prediction or response. Both training and inference have environmental costs. section 1.14 section 11.7
Several parts of an LLM’s operation matter:
- Training can run for weeks on large numbers of graphics processing units (GPUs) or tensor processing units (TPUs), specialized chips used for large-scale computation. section 1.14
- Data centers must also cool the heat-producing hardware. Cooling can require substantial amounts of water. section 11.3
- Inference continues after training. One study found that models can consume even more energy during this phase. section 1.14
Putting an estimate in perspective
One estimate put the carbon dioxide emitted from electricity used to train GPT-3 at 500 metric tons. That is an estimate for one training effort, not a known emissions figure for every LLM. Precise figures for most models remain unknown and depend on factors including the data center, hardware, model size, and architecture. section 1.14
Measurement helps make these costs visible. CodeCarbon can run alongside code and estimate the electricity use of a computer’s processor, memory, and GPUs. Other tools estimate energy use after training. Emissions may be reported as carbon dioxide equivalents (CO2eq), a common measure that includes greenhouse gases beyond carbon dioxide. Reporting is not yet widespread, so comparisons between models require care. section 11.3
Energy consumption is also a question of priorities: what public benefit justifies a model’s resource demands, and who can afford to build it? The hardware and money needed for large models favor well-resourced organizations. More systematic environmental reporting and efficient model development can help people assess that trade-off rather than treating larger models as an unquestioned goal. section 1.14 section 11.7
