Floating-point operation count
A floating-point operation count is the number of arithmetic operations involving values represented in floating-point format that a computation performs or requires.
Introduction to Generative AI, Second Edition
FLOPs are floating-point operations used to quantify computational work performed by a system or model.
- How it is used
- Uses training-operation totals to compare LLM compute and identify models subject to the EU AI Act’s systemic-risk obligations.
- How deep it goes
- Defines FLOPs and FLOPs/s, then gives scale comparisons and policy caveats without deriving operation counts.
- Examples
- Compares GPT-3’s roughly 10²³ training FLOPs with newer models near 10²⁵ and an EU threshold of 10²⁶.
- What you can do after
- Evaluate/understand training-compute comparisons and why a compute threshold cannot directly establish model capability.
Read how Introduction to Generative AI, Second Edition explains floating-point operation count →
Graph Neural Networks in Action
A floating-point operation is a numerical operation on floating-point values, with FLOPs denoting the count required by a computation.
- How it is used
- Uses FLOPs to compare computational scalability alongside memory, epoch time, and convergence speed.
- How deep it goes
- Defines the metric and demonstrates model profiling, but does not derive operation counts internally.
- Examples
- Uses a PyTorch model, a random input tensor, and THOP's profile function.
- What you can do after
- Apply via THOP's profile function to estimate a model's reported operation count.
Read how Graph Neural Networks in Action explains floating-point operation count →
Related concepts
- language modelmeasures · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- pretrainingconfigures · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
- floating-point arithmeticmeasures · Floating-point arithmetic is computation using finite-precision representations of real numbers, which can introduce rounding and range errors.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.