Pretrained model

As explained in Grokking AI Applications

A pretrained model is a model whose parameters have already been learned from an initial training corpus before later application or specialization.

A pretrained model has parameters learned during an initial training stage before those parameters are applied to new prompts during inference. Training starts with random parameters, then repeatedly adjusts them to reduce prediction errors. section 1.6

Its lifecycle has two practical stages:

• Training learns parameters from many examples, such as human-readable text covering varied subjects and contexts. section 1.6
• Inference applies the learned parameters to a prompt to generate a completion, rather than relearning the model each time. section 1.6

Figure 1.13 Enhancing a pretrained model through fine-tuning

Use the fine-tuning diagram to see how an already trained model can be enhanced for a later specialized use. section 1.6

How Grokking AI Applications covers it

How deep it goes

Explains parameter adjustment, backpropagation, training data, and inference, but stops short of detailed neural-network mathematics.

Examples

Uses GPT, GPT-3, sentence completion, a Rome prediction, and a 45-terabyte text dataset as concrete examples.

What it assumes

Assumes basic familiarity with prompts and completions while introducing neural-network training from random parameters.

What you can do after

Evaluate/understand how learned parameters support inference and why later specialization can begin from an existing model.

How it connects

Connects model training with inference, neural networks, parameters, backpropagation, and fine-tuning.

liveBookmanning.com

Read Grokking AI Applications on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonA generative AI model that has undergone initial training before being adapted or used for a task.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)Pretrained weights are model parameter values learned previously on another task or dataset and reused for later adaptation.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA pretrained model is a machine-learning model whose parameters have already been learned from prior training data.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA pretrained model is a machine-learning model whose parameters were learned previously on a dataset or task before subsequent use.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsA pretrained model is a model whose parameters were learned on earlier data before its application to a new task or dataset.
• 

  liveBookmanning.com
  Build AI-Enhanced Web AppsA pretrained model is a machine-learning model whose parameters have already been learned from training data before being adapted or used for an application.

Compare how 7 books explain pretrained model →

Related concepts

• foundational modelincludes · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
• SAMincludes · SAM, or Segment Anything Model, is a pretrained promptable computer-vision model that generates segmentation masks for objects in images.
• fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• pretrainingproduced by · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
• transfer learningused by · Transfer learning is a machine-learning technique that reuses knowledge learned from one task, domain, or dataset to improve learning or performance on another.
• feature engineeringuses · Feature engineering is the creation, transformation, or selection of input features from data to provide useful representations for analysis or modeling.

This explainer was generated by Manning's knowledge graph from the text of Grokking AI Applications. The section links point to the parts of the book it draws on.