GPT model

As explained in How Large Language Models Work

GPT is a family of generative pretrained transformer-based language models developed by OpenAI.

A GPT model is a generative language model: software that creates new text from patterns learned from earlier data. GPT means Generative Pretrained Transformer. section 1.0 section 1.1

Given a prompt, it generates an output shaped by its training and the requested task. It can write, rewrite, add detail, or complete text, but fluent output is not automatically accurate. section 1.1 section 1.0

The name points to three ideas:

• Generative: produces new content rather than merely retrieving a stored answer. section 1.1
• Pretrained: learns from data before being used for particular interactions or applications. section 1.6
• Transformer: uses a neural-network component associated with GPT models and modern generative AI. section 1.6

A prompt and an output

Prompting ChatGPT to write a haiku about snow falling on pines produces a novel haiku by drawing on learned patterns about haikus, snow, pines, and poetry. section 1.1

Figure 1.2 Generative AI takes some input (numbers, text, images) and produces a new output (usually text or images). Any combination of input or output options is possible, and the nature of the output depends on what the algorithm was trained for. It could be to add detail, rewrite something to be shorter, extrapolate missing portions, and more.

Trace how generative AI turns inputs such as text, images, or numbers into new outputs, with the result depending on training. section 1.1

GPT models are one family within the wider landscape of large language models. Their shared ideas apply across many systems, although commercial implementations may include private details and are not described down to code-level construction. section 1.6

How How Large Language Models Work covers it

How deep it goes

Defines generation, pretraining, and transformers at a high level, while stopping short of coding an LLM from scratch.

Examples

Uses a snow-and-pines haiku prompt and an input-output diagram covering text, images, and numbers.

What it assumes

Assumes no machine-learning background and builds terminology from everyday descriptions of language, data, and generated content.

What you can do after

Evaluate GPT-based tools more effectively by distinguishing generated fluency from accuracy, capability, limitation, and risk.

How it connects

Connects GPT models with generative AI, large language models, natural language processing, transformers, pretraining, and foundation models.

liveBookmanning.com

Read How Large Language Models Work on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  WordPress Power Toolkit: Harness AI to build next-level websitesGPT is a family of generative transformer language models developed by OpenAI for text, reasoning, programming, and multimodal tasks.
• 

  liveBookmanning.com
  Grokking AI ApplicationsGPT is a generative pretrained transformer language model that predicts likely continuations of text.
• 

  liveBookmanning.com
  The Complete Obsolete Guide to Generative AIA family of generative language models based on the Generative Pre-trained Transformer architecture.
• 

  liveBookmanning.com
  Sutskever's ListThe GPT family is a series of transformer-based autoregressive language models.
• 

  liveBookmanning.com
  Build a Large Language Model (From Scratch)A family of decoder-only generative pretrained transformer language models.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA family of generative pretrained Transformer language models referenced here as the host family that Codex is integrated into.
• 

  liveBookmanning.com
  LLMs in ProductionA GPT model is a generative pretrained transformer language model that produces text from an input context.
• 

  liveBookmanning.com
  Generative AI in ActionA family of generative pretrained transformer language models developed by OpenAI.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionGPT is a family of autoregressive Transformer language models that generate text by predicting successive tokens from preceding context.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionGPT is a family of autoregressive transformer language models developed by OpenAI and pretrained on large text corpora.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA decoder-only transformer language model that generates text by predicting the next token from left-context only.
• 

  liveBookmanning.com
  Build a Website with ChatGPTGPT is a family of generative pretrained transformer language models that generate text from input context.
• 

  liveBookmanning.com
  Elasticsearch in Action, Second EditionA generative pretrained transformer is a transformer-based neural-network architecture pretrained to generate or understand text.
• 

  liveBookmanning.com
  Inside AIGPT-3 is a generative language model developed by OpenAI for processing and generating human-language text.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA decoder-only transformer architecture used for autoregressive text generation without a separate encoder stack.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthGPT-3 is a large autoregressive transformer language model that generates text from preceding context.
• 

  liveBookmanning.com
  Build Python Web Apps with StreamlitGPT-5.1 is an OpenAI large language model in the GPT-5 family that supports configurable reasoning behavior.

Compare how 18 books explain GPT model →

Related concepts

• ChatGPTused by · ChatGPT is OpenAI’s conversational AI service, which lets users interact with generative AI models and related tools through natural-language prompts.
• decoder-only transformerkind of · A decoder-only transformer is a Transformer architecture without a separate encoder that processes preceding context to predict or generate subsequent sequence elements.
• deep learningkind of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• GPT-4contrasts with · GPT-4 is a generative large language model developed by OpenAI for understanding and generating text and other supported inputs and outputs.

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.