liveBook · Manning

GPT model

As explained in Fight Fraud with Machine Learning

GPT is a family of generative pretrained transformer language models trained to model and generate text.

A GPT model is a generative pretrained transformer: a deep-learning language model trained on huge volumes of internet text to model language. Its core objective is language modeling—estimating which word or word sequence is likely to follow preceding text. “Large” in large language model (LLM) refers to the immense volume of training data. ChatGPT is powered by a GPT model. section 5.0 section 5.2.1

Generation proceeds autoregressively. Given a prompt, the model assigns probabilities to possible next words, selects or samples a continuation, appends it to the sequence, and repeats. Training on extensive text lets such models learn probabilistic regularities about which words and sequences tend to occur together, enabling human-like text generation and conversation. section 5.2.1 section 5.2.1

Figure 5.9 The Generative Pretrained Transformer (GPT) model architecture, and how it auto-regressively performs text generation.

The architecture diagram helps connect the GPT name to its autoregressive generation behavior: each generated continuation becomes context for predicting what follows. section 5.2.1

Because they work with text, GPT-based LLMs can support tasks such as:

  • Translation, by producing text in another language from an input sequence. section 5.0
  • Summarization, question answering, and sentiment analysis. section 5.0
  • Email-phishing detection, using ChatGPT to classify emails, explain its decisions, and receive instructions intended to improve detection. section 5.2

Next-word language modeling

For the partial sentence “French is a beautiful …,” a trained language model can assign a higher probability to “language” than to “country.” The same next-word prediction process can be applied repeatedly to produce a longer response. A toy generation example can choose the most likely next word at every step, an approach called greedy search. section 5.2.1 section 5.2.1

For phishing detection, GPT-based interaction extends earlier rule-based and classical machine-learning approaches into a deep-learning workflow. The practical goal is not only to identify phishing emails, but also to ask for the reasoning behind a decision and instruct ChatGPT to become better at the detection task. section 5.0 section 5.2

How Fight Fraud with Machine Learning covers it

How deep it goes
Defines next-word language modeling and illustrates autoregressive generation, while directing from-scratch LLM construction elsewhere.
Examples
Illustrates next-word choices for “French is a beautiful …” and “It will,” plus a ChatGPT fraud conversation.
What you can do after
Apply it via ChatGPT to detect phishing emails, request decision interpretations, and instruct improved detection.
How it connects
Connects language modeling, large language models, transformers, ChatGPT, and email phishing detection.

Also explained in

Compare how 20 books explain GPT model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.