liveBook · Manning

Foundation model

As explained in Understanding AI

A foundation model is a broadly pretrained model that can be adapted to perform multiple downstream tasks or behaviors.

A foundational model is broadly pretrained on large amounts of text so it can continue language coherently, imitate styles, answer questions, and generate code. It learns these abilities by predicting what comes next rather than by receiving hand-written rules. section 2.2.1 section 2.3

That flexibility does not automatically make the model an assistant. Before fine-tuning, it may treat a prompt as text to continue, producing a question answer, web page, forum post, or story continuation unpredictably. section 2.2.2

The same prompt, different behavior

Give the same request to a foundational model and a fine-tuned conversational model. The foundational model may extend the surrounding text, while the conversational model interprets the input as an instruction and responds directly. Fine-tuning creates this behavioral shift through carefully curated prompt-response examples. section 2.2.2

Fine-tuning builds on the pretrained foundation with a smaller, higher-quality dataset. Later alignment training can shape preferred behavior and guardrails, while further domain training can produce specialized models for coding, medicine, law, or multimodal tasks. section 2.3

How Understanding AI covers it

How deep it goes
Explains next-token prediction and behavior before fine-tuning, but stops short of detailed training mathematics or implementation.
Examples
Uses coherent completions, style imitation, prompt-response pairs, and identical prompts compared across foundation and conversational models.
What it assumes
Assumes basic exposure to language models and introduces pretraining, fine-tuning, and alignment through plain-language contrasts.
What you can do after
Understand why pretraining alone produces text continuation rather than reliable instruction following.
How it connects
Connects pretraining with fine-tuning, conversational models, alignment training, guardrails, context windows, and specialized models.

Also explained in

Compare how 30 books explain foundation model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.