liveBook · Manning

Carbon footprint

As explained in Transfer Learning for Natural Language Processing

The greenhouse-gas emissions attributable to an activity, process, or product, commonly expressed as an equivalent amount of carbon dioxide.

A carbon footprint is the greenhouse-gas emissions attributable to an activity, process, or product, often expressed as an equivalent amount of carbon dioxide. For pretrained NLP models, it includes the environmental cost of training and repeated fine-tuning or hyperparameter optimization. section 11.4

Comparing model costs

Training BERT once was found to have a footprint equivalent to an average roundtrip flight between New York and San Francisco. With neural architecture search, one model deployment was estimated to cost as much as five average cars over their lifetimes. section 11.4

How Transfer Learning for Natural Language Processing covers it

How deep it goes
Connects repeated training, fine-tuning, hyperparameter optimization, and deployment to emissions through high-level comparisons.
Examples
Compares one BERT training run to a roundtrip flight and neural architecture search deployment to five cars’ lifetime footprints.
What you can do after
Evaluate pretrained models while taking their environmental costs into account.
How it connects
Links carbon footprint to pretrained language models, fine-tuning, hyperparameter optimization, neural architecture search, and parameter-efficient models.

Also explained in

Compare how 3 books explain carbon footprint →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Transfer Learning for Natural Language Processing. The section links point to the parts of the book it draws on.