Foundational model
A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
Inside Deep Learning: Math, Algorithms, Models
A model whose parameters were fitted on a source task or dataset before reuse.
- How it is used
- Uses source-task knowledge to build accurate target models faster and with fewer labeled examples.
- How deep it goes
- Explains weight and architecture reuse, final-layer replacement, freezing, warm starts, and PyTorch implementation tradeoffs.
- Examples
- Demonstrates ResNet18 image classification, frozen BERT text classification, a 256-item AG News corpus, and GRU baselines.
- What you can do after
- Apply it via PyTorch by loading pretrained weights, replacing an output layer, and choosing frozen, warm, or hybrid training.
Read how Inside Deep Learning: Math, Algorithms, Models explains foundational model →
Deep Learning with Python, Third Edition
A model trained beforehand on large-scale data and then reused for downstream tasks or analysis without starting from scratch.
- How it is used
- Positions broadly trained models as reusable starting points for downstream tasks instead of training each model from scratch.
- How deep it goes
- Defines foundation models and works through feature extraction and fine-tuning, while using pretrained weights rather than reproducing large-scale pretraining.
- Examples
- Uses ImageNet-pretrained Xception for image classification, RoBERTa for movie-review sentiment, and SAM for segmentation.
- What you can do after
- Apply it via KerasHub presets to load matching pretrained components, attach task-specific heads, and fine-tune where needed.
Read how Deep Learning with Python, Third Edition explains foundational model →
Deep Learning with R, Third Edition
A foundation model is a large pretrained model intended to support many downstream tasks with little or no task-specific retraining.
- How it is used
- Positions broad pretraining as a way to reuse one model across downstream tasks rather than train a separate model from scratch for each.
- How deep it goes
- Defines self-supervised pretraining and works through backbone reuse, classification heads, and fine-tuning; does not build a foundation model from scratch.
- Examples
- Uses RoBERTa for IMDb review classification and pretrained convolutional backbones for image classification and object detection.
- What you can do after
- Apply it via KerasHub by loading a pretrained backbone, attaching a task-specific head, and fine-tuning it.
Read how Deep Learning with R, Third Edition explains foundational model →
AI-Powered Search
A pretrained model is a model that has already been trained on general data before adaptation to a particular task or domain.
- How it is used
- Provides reusable models for query interpretation, embeddings, multimodal search, and generated search responses.
- How deep it goes
- Defines pretraining and adaptation, then demonstrates model reuse in a question-answering pipeline rather than training a foundation model from scratch.
- Examples
- Uses RoBERTa-derived question-answering guesses, human-corrected labels, and text-to-image search to make reuse concrete.
- What you can do after
- Apply it via a Hugging Face Transformers pipeline to generate candidate answers, then review them before treating them as labels.
Generative AI in Action
A pretrained model is a machine-learning model that has already been trained on an initial dataset before being adapted to another task or dataset.
- How it is used
- Provides a reusable starting model for enterprise generative-AI applications that must be connected to business workflows.
- How deep it goes
- Defines large-scale pretraining and adaptation, then outlines fine-tuning stages without deriving model-training mathematics.
- Examples
- Uses GPT-4 as an enterprise starting point and base LLMs as examples of pretrained text generators.
- What you can do after
- Evaluate/understand it by comparing model suitability and risks, then piloting its integration with a defined workflow.
Read how Generative AI in Action explains foundational model →
Time Series Forecasting Using Foundation Models
A foundation model is a machine learning model trained on large, diverse data and applicable to a wide variety of tasks, often through adaptation or fine-tuning.
- How it is used
- Applies one pretrained model across forecasting datasets and tasks, reducing the need to build and maintain separate data-specific models.
- How deep it goes
- Defines large-scale pretraining, reuse, zero-shot prediction, and fine-tuning, then evaluates frequency, horizon, capability, privacy, and resource limits.
- Examples
- Uses N-BEATS, TimeGPT, Lag-Llama, Chronos, Moirai, TimesFM, PromptCast, and Time-LLM for forecasting experiments and comparisons.
- What you can do after
- Evaluate and apply foundation forecasting models through zero-shot prediction, fine-tuning, API or package access, and metric-based comparison.
Read how Time Series Forecasting Using Foundation Models explains foundational model →
Deep Learning with PyTorch, Second Edition
A neural network already trained on prior data and reused for inference or further training.
- How it is used
- Provides trained weights for immediate inference and, in some cases, adaptation to new tasks or datasets.
- How deep it goes
- Defines pretrained weights and inference, contrasts task-specific recognition with zero-shot segmentation, and leaves Vision Transformer fine-tuning as an exercise.
- Examples
- Uses AlexNet image labels, Segment Anything object masks, and a pretrained Vision Transformer fine-tuning exercise.
- What you can do after
- Apply it via TorchVision for image recognition or use Segment Anything for zero-shot image segmentation.
Read how Deep Learning with PyTorch, Second Edition explains foundational model →
Machine Learning System Design
A machine learning model whose parameters were learned previously on a dataset or task before being applied or adapted to another task.
- How it is used
- Positions pretrained models as quick deep-learning baselines and large foundation models as sources of initial labels.
- How deep it goes
- Explains direct reuse, embeddings, zero-shot and few-shot use, and CLIP scoring without deriving model-training internals.
- Examples
- Illustrates ImageNet reuse for pet classification and CLIP scoring of text queries against images and descriptions.
- What you can do after
- Apply it via CLIP as an initial image-search scorer, then evaluate whether task-specific fine-tuning improves the baseline.
Read how Machine Learning System Design explains foundational model →
Delivering Machine Learning Projects: From design to deployment
A foundation model is a broadly trained machine learning model that can be adapted to support multiple downstream tasks.
- How it is used
- Uses pretrained models to explore unstructured data and create features for later ML tasks.
- How deep it goes
- Explains embeddings and gives model-selection and governance actions, without detailing model training internals.
- Examples
- Uses image and text model choices, embeddings of Shakespeare’s plays, and a nearest-neighbor heat map.
- What you can do after
- Apply it via EfficientNet or all-MiniLM-L12-v2 to generate embeddings, while tracking model approval and version consistency.
LLMs in Production
A foundation model is a broadly pretrained model that can be adapted to multiple downstream tasks or domains.
- How it is used
- Serves as the pretrained starting point for an LLM customized to a task or domain.
- How deep it goes
- Explains broad pretraining and subsequent finetuning, but does not detail foundation-model architecture.
- Examples
- Uses Wikipedia, books, and internet text as pretraining sources, then sentiment analysis, text classification, and question answering as target tasks.
- What you can do after
- Evaluate/understand whether a pretrained model is a suitable starting point before preparing data for finetuning.
How Large Language Models Work
A foundation model is a broadly trained model that can be adapted or used as a basis for many applications.
- How it is used
- Provides a reusable starting point for applications and task-specific models built from pretrained LLMs.
- How deep it goes
- Defines the pretraining-to-fine-tuning relationship but stops short of implementing an LLM from scratch.
- Examples
- Uses ChatGPT, Claude, Gemini, Llama, Copilot, EleutherAI models, and BigScience models as examples of the surrounding LLM landscape.
- What you can do after
- Apply it by fine-tuning a pretrained model into a task-specific system.
Read how How Large Language Models Work explains foundational model →
Hugging Face in Action
A pretrained model is a machine learning model whose parameters have already been learned from a training dataset before being adapted or used for a particular task.
- How it is used
- Deploys existing model knowledge for NLP, computer vision, and audio applications without requiring task-specific training from scratch.
- How deep it goes
- Explains reuse, discovery, inference, and fine-tuning, while stopping short of neural-network fundamentals and training-from-scratch methods.
- Examples
- Demonstrates DistilBERT sentiment classification, DETR object detection, Model Hub search, and device-aware sequence-classification loading.
- What you can do after
- Apply it via Hugging Face Transformers pipelines, Hosted Inference API, or task-specific model loaders.
Read how Hugging Face in Action explains foundational model →
Effective Data Analysis
A foundation model is a broadly pretrained model that can be adapted or prompted for many downstream tasks and domains.
- How it is used
- Supports general-purpose content generation and routine analytics workflows through prompts and integrated assistants.
- How deep it goes
- Defines pretraining, prompting, and general applicability but stops short of explaining model architecture or training internals.
- Examples
- Uses ChatGPT 3.5 and 4.0, analytics assistants, and customer-login summary prompts as examples.
- What you can do after
- Apply it via ChatGPT or analytics assistants to draft content and request exploratory summaries from data.
Read how Effective Data Analysis explains foundational model →
The AI Pocket Book
A preexisting model used as-is or with minimal adaptation instead of building one from scratch.
- How it is used
- Uses ready-made models for accessible AI applications, beginning with prompts or retrieval before considering parameter changes.
- How deep it goes
- Explains unaltered use, prompt adaptation, retrieval, and fine-tuning parameters, but stops short of training procedures.
- Examples
- Uses a world-record table prompt, few-shot demonstrations, RAG context, and Midjourney image prompts as concrete adaptations.
- What you can do after
- Apply it via prompts and retrieval, then evaluate whether fine-tuning is justified for the intended task.
Reinforcement Learning from Human Feedback
A foundation model is a broadly pretrained model that can be adapted to support multiple downstream tasks.
- How it is used
- Serves as the pretrained starting point that instruction tuning and RLHF reshape into a helpful question-answering model.
- How deep it goes
- Explains behavioral adaptation through instruction tuning, reward modeling, and RLHF, while deferring pretraining mechanics and model internals.
- Examples
- Contrasts an unhelpful boiling-point continuation with a direct instruction-tuned answer and cites an OLMoE post-training evaluation change.
- What you can do after
- Evaluate/understand why broad pretrained capability requires post-training to become reliable instruction-following behavior.
Read how Reinforcement Learning from Human Feedback explains foundational model →
Software Testing with Generative AI
A foundational model is a pretrained machine-learning model that provides a general base for further adaptation or specialization.
- How it is used
- Positions a pretrained model as a general base for products and further task-specific training.
- How deep it goes
- Defines the model and outlines fine-tuning, but defers detailed treatment to goals, data, tuning, and testing.
- Examples
- Uses GPT-3.5-Turbo, GPT-4o, ChatGPT, GitHub Copilot, Microsoft Bing AI, and Gemini variants as concrete examples.
- What you can do after
- Apply it via fine-tuning to adapt a model’s context, tone, or responses, then test the result.
Read how Software Testing with Generative AI explains foundational model →
Natural Language Processing in Action, Second Edition
A pretrained model is a model whose parameters were previously learned from a separate training corpus or task.
- How it is used
- Uses large-corpus pretraining as a reusable starting point for downstream applications and task-specific adaptation.
- How deep it goes
- Explains pretraining and fine-tuning procedurally, but stops short of detailing the underlying optimization internals.
- Examples
- Demonstrates CNN embeddings and GPT-2 adaptation using PyTorch datasets, text corpora, evaluation splits, and Hugging Face Trainer.
- What you can do after
- Apply it via Hugging Face Trainer to adapt GPT-2 to a focused text corpus and compare generated outputs.
Read how Natural Language Processing in Action, Second Edition explains foundational model →
Transformers in Action
A pretrained model is a machine-learning model whose parameters have already been learned from a broad training corpus before adaptation to a specific task.
- How it is used
- Deploys broad pretrained knowledge as a starting point for downstream language, vision, audio, or time-series tasks.
- How deep it goes
- Defines the category and contrasts related model types, but stops short of explaining training internals or parameter-update procedures.
- Examples
- Uses Hugging Face sentiment analysis, broad task adaptation, zero-shot use, few-shot use, and multiple transformer architecture variants.
- What you can do after
- Apply it via Hugging Face Transformers by selecting a pretrained model and adapting it to a specific labeled or prompt-based task.
Read how Transformers in Action explains foundational model →
Deep Learning with JAX
A model that has already been trained on a large corpus and can be reused directly or adapted to a new task.
- How it is used
- Uses existing models for language and image-generation tasks, beginning with prompting and escalating to adaptation when necessary.
- How deep it goes
- Explains prompting, pretraining, fine-tuning approaches, and cost tradeoffs, while deferring retrieval-augmented generation.
- Examples
- Names Llama, Gemma, GPT-J, GPT-Neo, T5, BERT, FinBERT, BioBERT, PEGASUS, and DALL·E Mini.
- What you can do after
- Apply pretrained models through Hugging Face Transformers and choose fine-tuning when prompting is insufficient.
Read how Deep Learning with JAX explains foundational model →
The Art of AI Product Development
A foundational model is a broadly trained model that can be adapted to support many downstream tasks and applications.
- How it is used
- Supports downstream applications by providing pretrained capabilities that AI engineers integrate into user-facing products.
- How deep it goes
- Defines adaptation and integration at a practical level, while stopping short of model-training internals or fine-grained optimization.
- Examples
- Uses prompts, AI-powered applications, production workflows, and unstructured-data pipelines as practical applications.
- What you can do after
- Apply it via pretrained models, prompts, APIs, and integrated AI workflows in production applications.
Read how The Art of AI Product Development explains foundational model →
Financial AI in Practice
A foundation model is a large, broadly trained model that can be adapted or guided for many downstream tasks.
- How it is used
- Positions foundation models as general-purpose engines to steer toward specific financial business problems.
- How deep it goes
- Defines the shift from task-specific training to AI engineering without detailing implementation steps.
- Examples
- Contrasts market-sentiment classification and research-report summarization as formerly separate modeling tasks.
- What you can do after
- Apply it by steering a general-purpose model safely and effectively toward a specific business problem.
Read how Financial AI in Practice explains foundational model →
Related concepts
- language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- pretrained modelkind of · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used or adapted for a subsequent task.
- instruct modelincludes · An instruction-tuned model is a pretrained model further trained to follow natural-language instructions and produce task-appropriate responses.
- feature engineeringused by · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.
- TimesFMincludes · TimesFM is a foundation model designed for deterministic time-series forecasting.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.