liveBook · Manning

Foundation model

As explained in Timeless Algorithms, The Foundational Ideas

A foundation model is a model trained on broad data at large scale and adaptable to multiple downstream tasks.

A foundation model is trained once on broad data at large scale, learning representations that can be refined, prompted, or otherwise adapted for multiple related tasks. This shifts learning away from estimating each model from scratch with task-specific data. section 11.7

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines large-scale broad-data training and downstream adaptation, without specifying an architecture or adaptation procedure.
What you can do after
Understand how broad pretraining can reduce reliance on task-specific training data.
How it connects
Links foundation models to self-supervised learning, representation learning, multimodal learning, and transformers.

Also explained in

Compare how 30 books explain foundation model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.