Energy efficiency measures the energy required to generate model output. After width pruning, energy use falls from 1.322 joules per token for the base model to 1.236 and 1.239 joules for the Wiki and SMS models, reported gains of 6.5% and 6.3%. Treat these figures as comparative baselines: shared-hardware workloads vary, and emissions estimates may not capture a data center’s full carbon footprint. section 5.2.6
Table 5.3 Energy efficiency comparison
|
Model
|
Energy
|
Joules / Token
|
Efficiency
|
Tokens Generated
|
|
Base
|
1908.42
|
1.322
|
|
1444
|
|
Wiki model
|
2544.24
|
1.236
|
6.5%
|
2059
|
|
SMS model
|
2473.31
|
1.239
|
6.3%
|
1997
|
Compare total energy, joules per token, and output counts for the base and pruned models. section 5.2.6
Table 5.2 Inference performance
|
Model
|
Tokens / sec
(WikiText)
|
Tokens / sec
(SMS)
|
Gains
|
|
Base
|
18.31
|
18.26
|
|
|
Wiki model
|
21.90
|
21.93
|
+19.6% / +20.1%
|
|
SMS model
|
21.83
|
21.89
|
+19.2% / +19.9%
|
Compare generation speed alongside energy use: the pruned models generate about 20% more tokens per second. section 5.2.6
