Foundation model
A foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
Building Specialized AI Systems
A machine-learning model trained on data before being adapted to a downstream task or domain.
- How it is used
- Provides the reusable starting point for fine-tuning language models into domain- and task-specific systems.
- How deep it goes
- Explains pretraining, selection criteria, and adapter-based adaptation, while deferring detailed training mechanics to fine-tuning methods.
- Examples
- Uses financial report summarization, biomedical question answering, customer-service replies, and swappable finance and customer-support adapters.
- What you can do after
- Evaluate and select a starting model by task fit, domain exposure, compute requirements, and deployment constraints.
Read how Building Specialized AI Systems explains foundation model →
Inside Deep Learning: Math, Algorithms, Models
A model whose parameters were fitted on a source task or dataset before reuse.
- How it is used
- Provides trained weights as a starting point for a new task when labeled data or training time is limited.
- How deep it goes
- Works through warm, frozen, and partially frozen reuse with PyTorch comparisons, without detailing how the original pretraining was performed.
- Examples
- Compares fresh, warm, and frozen ResNet-18 models, then uses a frozen BERT classifier against a GRU baseline.
- What you can do after
- Apply it via PyTorch by replacing an output layer and choosing whether pretrained weights remain trainable.
Read how Inside Deep Learning: Math, Algorithms, Models explains foundation model →
Deep Learning with Python, Third Edition
A model trained beforehand on large-scale data and then reused for downstream tasks or analysis without starting from scratch.
- How it is used
- Uses reusable pretrained models to tackle downstream tasks without repeating large-scale training.
- How deep it goes
- Defines foundation models and works through feature extraction and fine-tuning, while deferring Xception architecture details and batch-normalization rationale.
- Examples
- Uses Xception for dogs-versus-cats classification, Segment Anything for segmentation, and RoBERTa for IMDb classification.
- What you can do after
- Applies pretrained models using KerasHub presets, matching preprocessing, and task-specific classifier heads.
Read how Deep Learning with Python, Third Edition explains foundation model →
Deep Learning with R, Third Edition
A foundation model is a large pretrained model intended to support many downstream tasks with little or no task-specific retraining.
- How it is used
- Introduces foundation models as reusable pretrained systems for generative AI and varied downstream tasks.
- How deep it goes
- Explains self-supervised objectives and reuse, then demonstrates prompting and fine-tuning pretrained models.
- Examples
- Uses multimodal Gemma image questions and RoBERTa IMDb review classification to illustrate pretrained model reuse.
- What you can do after
- Apply pretrained models using KerasHub for review classification and multimodal prompting.
Read how Deep Learning with R, Third Edition explains foundation model →
AI-Powered Search
A pretrained model is a model that has already been trained on general data before adaptation to a particular task or domain.
- How it is used
- Uses foundation models as reusable components for query understanding, semantic search, question answering, and search-result summarization.
- How deep it goes
- Defines pretraining and adaptation, then demonstrates model-backed search workflows without detailing how to train a foundation model from scratch.
- Examples
- Uses RoBERTa-based question answering to generate a silver set and semantic text similarity data to illustrate search-oriented fine-tuning.
- What you can do after
- Apply it via the Hugging Face Transformers question-answering pipeline to generate candidate labels for human review.
Generative AI in Action
A pretrained model is a machine-learning model that has already been trained on an initial dataset before being adapted to another task or dataset.
- How it is used
- Positions foundational models as reusable starting points for enterprise generative AI applications.
- How deep it goes
- Explains pretraining and adaptation choices, then outlines fine-tuning stages without mathematical derivation.
- Examples
- Uses GPT-3’s DaVinci as a base LLM and sentiment analysis as a fine-tuning task.
- What you can do after
- Evaluate and understand adaptation choices and their trade-offs for enterprise tasks.
Read how Generative AI in Action explains foundation model →
Time Series Forecasting Using Foundation Models
A foundation model is a machine learning model trained on large, diverse data and applicable to a wide variety of tasks, often through adaptation or fine-tuning.
- How it is used
- Applies one pretrained model across forecasting datasets and tasks, reducing the need to build and maintain separate data-specific models.
- How deep it goes
- Defines large-scale pretraining, reuse, zero-shot prediction, and fine-tuning, then evaluates frequency, horizon, capability, privacy, and resource limits.
- Examples
- Uses N-BEATS, TimeGPT, Lag-Llama, Chronos, Moirai, TimesFM, PromptCast, and Time-LLM for forecasting experiments and comparisons.
- What you can do after
- Evaluate and apply foundation forecasting models through zero-shot prediction, fine-tuning, API or package access, and metric-based comparison.
Read how Time Series Forecasting Using Foundation Models explains foundation model →
Deep Learning with PyTorch, Second Edition
A neural network already trained on prior data and reused for inference or further training.
- How it is used
- Uses pretrained networks for image recognition and a foundational model for prompted segmentation of CT nodules.
- How deep it goes
- Explains loading pretrained weights and inference, introduces zero-shot segmentation, and assigns fine-tuning as an exercise.
- Examples
- Uses ImageNet classification, a Vision Transformer fine-tuning exercise, and Segment Anything for CT nodule segmentation.
- What you can do after
- Apply pretrained models via TorchVision and practice fine-tuning a Vision Transformer on a small dataset.
Read how Deep Learning with PyTorch, Second Edition explains foundation model →
Machine Learning System Design
A machine learning model whose parameters were learned previously on a dataset or task before being applied or adapted to another task.
- How it is used
- Uses pretrained models as deep learning baselines and foundation models to generate initial labels.
- How deep it goes
- Explains direct reuse, embeddings, and fine-tuning through a retrieval baseline, without detailing training algorithms.
- Examples
- Uses ImageNet features for custom image classification and CLIP encoders for image-search relevance scoring.
- What you can do after
- Evaluate pretrained-model reuse against simpler baselines using accuracy, development effort, interpretability, and computation time.
Read how Machine Learning System Design explains foundation model →
Delivering Machine Learning Projects: From design to deployment
A foundation model is a broadly trained machine learning model that can be adapted to support multiple downstream tasks.
- How it is used
- Uses pretrained models to explore unstructured data and create features for downstream modeling.
- How deep it goes
- Explains embedding-based exploration and model governance without detailing how foundational models are trained.
- Examples
- Uses Shakespeare’s plays to illustrate sentence embeddings, similarity, novelty, and a concept graph.
- What you can do after
- Apply all-MiniLM-L12-v2 to generate text embeddings for exploratory analysis and downstream features.
Read how Delivering Machine Learning Projects: From design to deployment explains foundation model →
LLMs in Production
A foundation model is a broadly pretrained model that can be adapted to multiple downstream tasks or domains.
- How it is used
- Positions a pretrained base model as the starting point for a task-specific LLM rather than requiring training from scratch.
- How deep it goes
- Explains pretraining, fine-tuning, and prompting as stages, contrasting their purposes and resource requirements.
- Examples
- Uses sentiment analysis, text classification, and question answering as task-specific adaptation examples.
- What you can do after
- Evaluate potential starting models using their pretraining history and the resource trade-offs of fine-tuning versus training from scratch.
Rearchitecting LLMs
A foundation model is a broadly pretrained model intended to support adaptation to many downstream tasks.
- How it is used
- Uses a base model as the starting point for task adaptation or general-purpose size reduction.
- How deep it goes
- Outlines baseline evaluation, depth pruning, impact measurement, and knowledge recovery; defers task-specific optimization.
- Examples
- Uses Gemma 3 270M, model inspection, and benchmark evaluations to establish a baseline before pruning.
- What you can do after
- Evaluate/understand a base model against a baseline before measuring the effects of structural changes.
Build AI Drug Discovery Pipelines
A foundation model is a broadly pretrained model that can support multiple downstream tasks through adaptation or reuse of its learned representations.
- How it is used
- Supports downstream drug-discovery tasks by reusing representations learned from large unlabeled biological datasets.
- How deep it goes
- Explains pretrained representations, transfer learning, and multimodal integration, but does not provide implementation procedures or model-training code.
- Examples
- Uses ESM-2, MegaMolBART, LucaOne, ProCyon, Evo 2, and BioNeMo-integrated drug-discovery workflows as examples.
- What you can do after
- Apply pretrained embeddings or fine-tuning to downstream biological predictions when task-specific labeled data is limited.
Read how Build AI Drug Discovery Pipelines explains foundation model →
How Large Language Models Work
A foundation model is a broadly trained model that can be adapted or used as a basis for many applications.
- How it is used
- Provides a reusable starting point for applications and task-specific models built from pretrained LLMs.
- How deep it goes
- Defines the pretraining-to-fine-tuning relationship but stops short of implementing an LLM from scratch.
- Examples
- Uses ChatGPT, Claude, Gemini, Llama, Copilot, EleutherAI models, and BigScience models as examples of the surrounding LLM landscape.
- What you can do after
- Apply it by fine-tuning a pretrained model into a task-specific system.
Read how How Large Language Models Work explains foundation model →
Retrieval Augmented Generation, The Foundational Ideas
A foundation model is a broadly trained model that can be adapted to multiple downstream tasks.
- How it is used
- Frames retrieval-augmented pretraining as a reusable base for downstream language tasks.
- How deep it goes
- Explains two training stages, joint retriever-reader learning, and periodic index rebuilding, but not optimization internals.
- Examples
- Uses Wikipedia or CC-News pretraining and Natural Questions fine-tuning for open-domain question answering.
- What you can do after
- Apply it via supervised fine-tuning to specialize general retrieval-augmented knowledge for a downstream task.
Read how Retrieval Augmented Generation, The Foundational Ideas explains foundation model →
Hugging Face in Action
A pretrained model is a machine learning model whose parameters have already been learned from a training dataset before being adapted or used for a particular task.
- How it is used
- Deploys existing model knowledge for NLP, computer vision, and audio applications without requiring task-specific training from scratch.
- How deep it goes
- Explains reuse, discovery, inference, and fine-tuning, while stopping short of neural-network fundamentals and training-from-scratch methods.
- Examples
- Demonstrates DistilBERT sentiment classification, DETR object detection, Model Hub search, and device-aware sequence-classification loading.
- What you can do after
- Apply it via Hugging Face Transformers pipelines, Hosted Inference API, or task-specific model loaders.
Effective Data Analysis
A foundation model is a broadly pretrained model that can be adapted or prompted for many downstream tasks and domains.
- How it is used
- Supports general-purpose content generation and routine analytics workflows through prompts and integrated assistants.
- How deep it goes
- Defines pretraining, prompting, and general applicability but stops short of explaining model architecture or training internals.
- Examples
- Uses ChatGPT 3.5 and 4.0, analytics assistants, and customer-login summary prompts as examples.
- What you can do after
- Apply it via ChatGPT or analytics assistants to draft content and request exploratory summaries from data.
Read how Effective Data Analysis explains foundation model →
The AI Pocket Book
A preexisting model used as-is or with minimal adaptation instead of building one from scratch.
- How it is used
- Uses ready-made models for accessible AI applications, beginning with prompts or retrieval before considering parameter changes.
- How deep it goes
- Explains unaltered use, prompt adaptation, retrieval, and fine-tuning parameters, but stops short of training procedures.
- Examples
- Uses a world-record table prompt, few-shot demonstrations, RAG context, and Midjourney image prompts as concrete adaptations.
- What you can do after
- Apply it via prompts and retrieval, then evaluate whether fine-tuning is justified for the intended task.
Reinforcement Learning from Human Feedback
A foundation model is a broadly pretrained model that can be adapted to support multiple downstream tasks.
- How it is used
- Uses base language models as the starting point for adapting pretrained capabilities to useful, instruction-following behavior.
- How deep it goes
- Explains completion behavior and outlines a three-stage RLHF recipe without detailing model architecture or optimization internals.
- Examples
- Contrasts answers to a boiling-point question and compares benchmark averages across two OLMoE Instruct versions.
- What you can do after
- Evaluate and understand how adaptation can change a base model’s responses and benchmark performance.
Read how Reinforcement Learning from Human Feedback explains foundation model →
LLM Customization and Fine-Tuning
The original pretrained model whose weights are kept fixed while LoRA learns an additive adaptation.
- How it is used
- Frames the starting model as a dependency in an adaptation lifecycle and a choice constrained by enterprise deployment needs.
- How deep it goes
- Explains selection across task, latency, hardware, and tooling constraints, and distinguishes pretrained-only from instruction-tuned starting models.
- Examples
- Uses Qwen3-4B-Instruct-2507 and a ticket-triage classifier to illustrate the starting model and prompt consistency.
- What you can do after
- Evaluate starting models against task headroom, latency, hardware, tooling, and available training data.
Read how LLM Customization and Fine-Tuning explains foundation model →
Software Testing with Generative AI
A foundational model is a pretrained machine-learning model that provides a general base for further adaptation or specialization.
- How it is used
- Positions a pretrained model as a general base for products and further task-specific training.
- How deep it goes
- Defines the model and outlines fine-tuning, but defers detailed treatment to goals, data, tuning, and testing.
- Examples
- Uses GPT-3.5-Turbo, GPT-4o, ChatGPT, GitHub Copilot, Microsoft Bing AI, and Gemini variants as concrete examples.
- What you can do after
- Apply it via fine-tuning to adapt a model’s context, tone, or responses, then test the result.
Read how Software Testing with Generative AI explains foundation model →
Understanding AI
A foundation model is a broadly pretrained model that can be adapted to perform multiple downstream tasks or behaviors.
- How it is used
- Explains the pretrained base that later becomes an instruction-following, aligned, or specialized model.
- How deep it goes
- Explains next-token prediction and behavior before fine-tuning, but stops short of detailed training mathematics or implementation.
- Examples
- Uses coherent completions, style imitation, prompt-response pairs, and identical prompts compared across foundation and conversational models.
- What you can do after
- Understand why pretraining alone produces text continuation rather than reliable instruction following.
Natural Language Processing in Action, Second Edition
A pretrained model is a model whose parameters were previously learned from a separate training corpus or task.
- How it is used
- Uses large-corpus pretraining as a reusable starting point for downstream applications and task-specific adaptation.
- How deep it goes
- Explains pretraining and fine-tuning procedurally, but stops short of detailing the underlying optimization internals.
- Examples
- Demonstrates CNN embeddings and GPT-2 adaptation using PyTorch datasets, text corpora, evaluation splits, and Hugging Face Trainer.
- What you can do after
- Apply it via Hugging Face Trainer to adapt GPT-2 to a focused text corpus and compare generated outputs.
Read how Natural Language Processing in Action, Second Edition explains foundation model →
Transformers in Action
A pretrained model is a machine-learning model whose parameters have already been learned from a broad training corpus before adaptation to a specific task.
- How it is used
- Deploys broad pretrained knowledge as a starting point for downstream language, vision, audio, or time-series tasks.
- How deep it goes
- Defines the category and contrasts related model types, but stops short of explaining training internals or parameter-update procedures.
- Examples
- Uses Hugging Face sentiment analysis, broad task adaptation, zero-shot use, few-shot use, and multiple transformer architecture variants.
- What you can do after
- Apply it via Hugging Face Transformers by selecting a pretrained model and adapting it to a specific labeled or prompt-based task.
Deep Learning with JAX
A model that has already been trained on a large corpus and can be reused directly or adapted to a new task.
- How it is used
- Uses existing models for language and image-generation tasks, beginning with prompting and escalating to adaptation when necessary.
- How deep it goes
- Explains prompting, pretraining, fine-tuning approaches, and cost tradeoffs, while deferring retrieval-augmented generation.
- Examples
- Names Llama, Gemma, GPT-J, GPT-Neo, T5, BERT, FinBERT, BioBERT, PEGASUS, and DALL·E Mini.
- What you can do after
- Apply pretrained models through Hugging Face Transformers and choose fine-tuning when prompting is insufficient.
Timeless Algorithms, The Foundational Ideas
A foundation model is a model trained on broad data at large scale and adaptable to multiple downstream tasks.
- How it is used
- Reuses broadly learned representations as a starting point for multiple related tasks.
- How deep it goes
- Defines large-scale broad-data training and downstream adaptation, without specifying an architecture or adaptation procedure.
- What you can do after
- Understand how broad pretraining can reduce reliance on task-specific training data.
Read how Timeless Algorithms, The Foundational Ideas explains foundation model →
Build a DeepSeek Model (From Scratch)
A base model is a pretrained model whose existing parameters and capabilities provide the foundation for subsequent adaptation or distillation.
- How it is used
- Guides foundation selection for knowledge distillation by matching pre-existing capabilities to the target task.
- How deep it goes
- Compares specialized and general bases through model variants and benchmarks, without detailing pretraining internals.
- Examples
- Uses Qwen2.5-Math, Qwen2.5, Llama 3, and six DeepSeek distilled model variants as concrete comparisons.
- What you can do after
- Evaluate candidate foundations against target capabilities and expected distilled-model performance.
Read how Build a DeepSeek Model (From Scratch) explains foundation model →
Local Image Generation
A base model is the pretrained model selected as the foundation for further adaptation or fine-tuning.
- How it is used
- Selects the model foundation for fine-tuning and supports choosing where the resulting model is saved.
- How deep it goes
- Explains model selection, output destinations, and precision trade-offs, then stops short of detailed model internals.
- Examples
- Uses Stable Diffusion 1.5, alternative fine-tunes, and checkpoint iterations whose human images improve across 10, 20, and 30 epochs.
- What you can do after
- Operate/configure a fine-tuning run by selecting a base model, output path, and weight and output data types.
The Art of AI Product Development
A foundational model is a broadly trained model that can be adapted to support many downstream tasks and applications.
- How it is used
- Supports downstream applications by providing pretrained capabilities that AI engineers integrate into user-facing products.
- How deep it goes
- Defines adaptation and integration at a practical level, while stopping short of model-training internals or fine-grained optimization.
- Examples
- Uses prompts, AI-powered applications, production workflows, and unstructured-data pipelines as practical applications.
- What you can do after
- Apply it via pretrained models, prompts, APIs, and integrated AI workflows in production applications.
Read how The Art of AI Product Development explains foundation model →
Financial AI in Practice
A foundation model is a large, broadly trained model that can be adapted or guided for many downstream tasks.
- How it is used
- Positions foundation models as general-purpose engines to steer toward specific financial business problems.
- How deep it goes
- Defines the shift from task-specific training to AI engineering without detailing implementation steps.
- Examples
- Contrasts market-sentiment classification and research-report summarization as formerly separate modeling tasks.
- What you can do after
- Apply it by steering a general-purpose model safely and effectively toward a specific business problem.
Read how Financial AI in Practice explains foundation model →
Also covered in
- Math and Architectures of Deep Learning
- Build a Reasoning Model (From Scratch)
- Build a Large Language Model (From Scratch)
- Machines that ThinkIn early access (MEAP)
- Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
- Essential GraphRAG
- Agent Design PatternsIn early access (MEAP)
- Effective Conversational AI
- Microsoft Azure in Action
- Introduction to Generative AI, Second Edition
- Digital Twins in Action
- Causal AI
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
- parameterA parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
Related concepts
- language modelkind of · A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- pretrained modelkind of · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used for inference or adapted to a later task.
- instruct modelincludes · An instruction-tuned model is a pretrained model further trained to follow natural-language instructions and produce task-appropriate responses.
- feature engineeringused by · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
- student modelcontrasts with · A student model is a model trained with guidance from a teacher model to approximate its outputs, behavior, or learned capabilities.
Questions about foundation model
What is foundation model?
A foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
Which Manning books explain foundation model?
Building Specialized AI Systems, Inside Deep Learning: Math, Algorithms, Models, Deep Learning with Python, Third Edition, Deep Learning with R, Third Edition, AI-Powered Search, Generative AI in Action, Time Series Forecasting Using Foundation Models, Deep Learning with PyTorch, Second Edition and 22 more.
What should I know before learning foundation model?
Manning books that teach foundation model usually introduce language model, fine-tuning, classifier and parameter first.
What is the difference between foundation model and student model?
A foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks. A student model is a model trained with guidance from a teacher model to approximate its outputs, behavior, or learned capabilities.
What kinds of foundation model are there?
Kinds of foundation model include instruct model.
What is foundation model a kind of?
Foundation model is a kind of language model. A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





























