liveBook · Manning

Energy consumption

As explained in Rearchitecting LLMs

Energy consumption is the quantity of electrical energy used by a computation or hardware system.

For model inference, compare energy use as joules per generated token. Measurements with CodeCarbon show lower per-token energy for pruned models, but shared hardware and workload variability make the results comparative baselines, not absolute metrics. section 5.2.6

Table 5.3 Energy efficiency comparison
Model
Energy
Joules / Token
Efficiency
 
Tokens Generated
Base
1908.42
1.322
 
1444
Wiki model
2544.24
1.236
6.5%
2059
SMS model
2473.31
1.239
6.3%
1997

Compare energy per token for the base and pruned models. section 5.2.6

How Rearchitecting LLMs covers it

How deep it goes
Reports per-token energy measurements and comparative results, while deferring evaluation-system details to appendices.
Examples
Compares base and pruned models at 1.322 versus 1.236–1.239 joules per token.
What you can do after
Evaluate pruning variants by comparing measured joules per generated token as comparative baselines.
How it connects
Connects energy consumption to width pruning and inference performance.

Also explained in

Compare how 7 books explain energy consumption →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.