A carbon footprint is the greenhouse-gas emissions attributable to an activity, process, or product, often expressed as an equivalent amount of carbon dioxide. For pretrained NLP models, it includes the environmental cost of training and repeated fine-tuning or hyperparameter optimization. section 11.4
Comparing model costs
Training BERT once was found to have a footprint equivalent to an average roundtrip flight between New York and San Francisco. With neural architecture search, one model deployment was estimated to cost as much as five average cars over their lifetimes. section 11.4


