liveBook · Manning

Energy efficiency

As explained in Rearchitecting LLMs

Energy efficiency is the amount of energy required to perform a computational workload or produce a unit of model output.

Energy efficiency measures the energy required to generate model output. After width pruning, energy use falls from 1.322 joules per token for the base model to 1.236 and 1.239 joules for the Wiki and SMS models, reported gains of 6.5% and 6.3%. Treat these figures as comparative baselines: shared-hardware workloads vary, and emissions estimates may not capture a data center’s full carbon footprint. section 5.2.6

Table 5.3 Energy efficiency comparison
Model
Energy
Joules / Token
Efficiency
 
Tokens Generated
Base
1908.42
1.322
 
1444
Wiki model
2544.24
1.236
6.5%
2059
SMS model
2473.31
1.239
6.3%
1997

Compare total energy, joules per token, and output counts for the base and pruned models. section 5.2.6

Table 5.2 Inference performance
Model
Tokens / sec
(WikiText)
Tokens / sec
(SMS)
Gains
Base
18.31
18.26
 
Wiki model
21.90
21.93
+19.6% / +20.1%
SMS model
21.83
21.89
+19.2% / +19.9%

Compare generation speed alongside energy use: the pruned models generate about 20% more tokens per second. section 5.2.6

How Rearchitecting LLMs covers it

How deep it goes
Measures joules per token with CodeCarbon and reports comparative results, while deferring evaluation-system details to appendices.
Examples
Compares base, Wiki, and SMS models at 1.322, 1.236, and 1.239 joules per token.
What you can do after
Evaluate pruned-model energy use against a baseline, treating reported figures as comparative rather than absolute.
How it connects
Connects energy efficiency with width pruning, inference performance, and calibration datasets.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.