liveBook · Manning

Chapter 4: Large language models

What you will learn in chapter 4 of Machines that Think

This introduction frames large language models as scaled statistical systems whose architecture, capacity, multimodal designs, and costs shape both their apparent intelligence and who can build them.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction frames large language models as scaled statistical systems whose architecture, capacity, multimodal designs, and costs shape both their apparent intelligence and who can build them.

Key terms: large language models, statistical learning, explicit model of meaning, reasoning, machine intelligence, Transformers, sequential constraints, parameters, capacity, data

4.1 The transformers architecture

This section introduces transformers as the architecture behind modern large language models, highlighting their whole-input token interactions, scalability, and need for supporting mechanisms.

Key terms: transformers, tokens, representations, broader context, scale, order, combine information, stable learning, transformer architecture, large language models

4.1.1 Attention was all we needed

This section introduces the Transformer’s attention-centered redesign of language modeling and explains why it enabled scalable large language models.

Key terms: Transformer, attention, attention mechanisms, repeating components, tokens, sequence, long-range relationships, parallel processing, fixed route, architectural constraints

4.1.2 Providing a sense of order

This section explains how Transformers supply token order to attention through positional encodings, including the original sinusoidal approach.

Key terms: attention mechanisms, recurrent networks, input sequence, token order, positional encoding, token embedding, Transformer architecture, representation, attention, sinusoidal encoding

4.1.3 Listening to the self

This section explains how self-attention contextualizes every token by scoring its relationships to all other tokens and combining their information.

Key terms: self-attention, token representation, contextualized representation, learned projections, trainable weights, query, key, value, dot product, relevance scores

4.1.4 Multiple heads are better than one

This section explains how multi-head attention gives a transformer several complementary ways to interpret relationships in the same sequence and combines them into a richer representation.

Key terms: attention head, single attention head, multi-head attention, attention heads, learned projections, queries, keys, values, head outputs, combined representation

4.1.5 Making depth trainable

This section explains how residual connections and layer normalization preserve information and stabilize training so Transformer depth can scale.

Key terms: residual connections, layer normalization, feedforward layers, contextualized representations, depth, input, output, incremental updates, representation, recentering

4.1.6 One block, many behaviors

This section explains how identical Transformer blocks serve as encoders or decoders through different attention permissions, enabling full-context processing and step-by-step text generation.

Key terms: Transformer block, encoder, decoder, autoencoder, self-attention, feedforward layers, <start>, token, next-token prediction, input sequence

4.1.7 Connecting representations

This section explains how cross-attention connects an encoder’s full-input representations to a decoder’s causal, token-by-token generation.

Key terms: cross-attention, queries, keys, values, encoder output, decoder, causal structure, masked attention, future tokens, encoder representations

4.1.8 From blocks to outputs

This section explains how Transformer representations become token outputs, compares the main Transformer model families, and shows why decoder-only next-token prediction now underlies most language systems.

Key terms: contextualized representation, linear layer, softmax, probability, vocabulary, next token, encoder-only, encoder-decoder, decoder-only, BERT

4.2 What makes a language model “large”

This section introduces unprecedented scale as the defining feature of large language models and previews the practical forms that scale takes.

Key terms: large language models, scale, unprecedented scale, parameter counts, data exposure, architectural conditions, computational conditions, training

4.2.1 Where the learning fits

This section explains parameter count as a measure of language-model capacity, how Transformer-enabled scale changed what counts as large, and why more parameters alone do not ensure better performance.

Key terms: parameters, learned numerical weights, capacity, constraints, internal representations, width, depth, features, partition the space of possible inputs, recurrent architectures

4.2.2 Large models through broad exposure

This section explains how broad, diverse training data fills an LLM’s representational capacity and how balanced growth in model size, data, and computation produces predictable performance gains.

Key terms: training data, vast text collections, domains, styles, purposes, representational space, learning signals, infrequent patterns, contextual cues, regularities

4.2.3 Scale comes with demands

This section explains why Transformer architecture makes large-scale language-model training practical through parallel computation, stable deep signal flow, and distance-independent attention.

Key terms: training feasibility, parameter counts, broad exposure, computation, memory, hardware coordination, Transformer, recurrence, sequential bottleneck, parallel computation

4.3 From large to usable

This section introduces the tension between training language models at scale and making them practical to deploy in real-world settings.

Key terms: large language models, scale, large-scale training, deployment, real-world limits, usable capability, reshaped, constrained, compressed

4.3.1 Learning before specialization

This section explains why large language models are pretrained broadly and then specialized through targeted adaptation rather than trained from scratch for every task.

Key terms: pretraining, general predictive objective, task-specific supervision, parameters, full fine-tuning, constrained adaptation, partial adaptation, frozen model, weights, trainable components

4.3.2 Adapting from within

This section explains how Low-Rank Adaptation (LoRA) efficiently specializes pretrained language models through small internal trainable corrections instead of full fine-tuning.

Key terms: full fine-tuning, selective approach, adaptation, Low-Rank Adaptation, LoRA, pretrained weights, frozen weights, linear projections, attention layers, task-specific adaptations

4.3.3 Transferring scale

This section explains knowledge distillation as a way to transfer a large model’s behavior into a smaller model that is practical to deploy under resource constraints.

Key terms: latency, memory, energy use, cost, deployment, compact and efficient forms, knowledge distillation, teacher, student, pretrained model

4.3.4 Compressing computation

This section explains how quantization compresses a model’s numeric execution to make deployment faster and cheaper while managing quality loss.

Key terms: quantization, parameters, activations, high-precision floating-point values, lower-precision numbers, rescaling, memory usage, latency, energy costs, approximation error

4.4 Somebody call the expert

This section introduces selective-activation architectures as a way to scale language models more efficiently than uniformly enlarging dense models.

Key terms: dense models, computation, representational capacity, uniform scaling, selective activation, specialize, internal differentiation, capacity

4.4.1 Experts needed

This section introduces Mixture of Experts layers as a way to scale language-model capacity efficiently by routing each token to only a few specialized subnetworks.

Key terms: dense architectures, depth, width, representational capacity, parameters, Mixture of Experts, MoE layer, feedforward block, experts, gating network

4.4.2 Learning to specialize

This section explains how mixture-of-experts models learn expert specialization through gated token routing, selective learning signals, and balanced expert use.

Key terms: gating network, routing, experts, token’s current representation, scores, weights, specialization, learning signal, final loss, feedback loop

4.4.3 Global scale, selective growth

This section explains how sparsely activated expert layers expand model capacity without proportional per-token computation, while introducing irregular hardware workloads and therefore being used selectively within Transformers.

Key terms: selective activation, experts, capacity, representational power, per-token computation, routing, tokens, batch, uneven workloads, scheduling

4.5 Beyond text: multimodality

This section introduces multimodal language models, which combine text with perceptual data and use attention to integrate different modalities without changing their data-driven learning basis.

Key terms: multimodal models, language models, images, audio, video, indirect record of the world, perceptual signals, align, language, attention

4.5.1 Worlds beyond words

This section explains why text-only language models inherit mediated descriptions rather than direct experience, and how multimodal training broadens the evidence they can use.

Key terms: text, abstraction, symbols, sensory details, shared background knowledge, mediated through language, linguistic fluency, perception, grounding, non-linguistic evidence

4.5.2 A shared space

This section explains how multimodal models encode different signals into a shared vector space and learn cross-modal relationships through contrastive training.

Key terms: shared representational space, common geometric space, vector representations, semantic relatedness, multimodal models, modality-specific encoders, text tokens, image patches, audio segments, visual features

4.5.3 Architectures for multimodal fusion

This section compares ways multimodal models combine modality-specific representations and explains how Transformer cross-attention enables selective interaction among text, images, and audio.

Key terms: late fusion, early fusion, encoding, shared layers, modalities, modality-specific encoders, shared Transformer backbone, cross-attention, textual prompt, visual features

4.5.4 Grounding without causation

This section explains that multimodal models connect language with perception through statistical co-occurrence, which can look like grounding but does not give them causal understanding of the world.

Key terms: multimodal models, linguistic frame, captions, transcripts, descriptions, conditioning, multimodal corpora, shared structure, co-occurrence, scale

4.6 The cost of intelligence

This section introduces the material and economic demands that make large language model capabilities inseparable from computation, energy, infrastructure, and resource constraints.

Key terms: statistical pattern recognition, computation, energy, infrastructure, cost, artifacts, training, deployment, efficiency, resource constraints

4.6.1 Training at scale

This section explains why training frontier language models is extraordinarily costly, technically demanding, energy-intensive, and difficult to repeat.

Key terms: training, parameters, data exposure, optimization steps, data preparation, inference, forward pass, backward pass, gradients, accelerators

4.6.2 The daily cost of intelligence

This section explains why serving a trained large language model creates continuous, latency-sensitive inference costs that shape deployment and product decisions.

Key terms: inference, forward pass, parameters, numerical operations, latency-sensitive, throughput, availability, peak demand, high-bandwidth memory, replicated across regions

4.6.3 Powering intelligence

This section explains why efficiency improvements cannot offset rising AI energy demand and how energy constraints shape model and deployment design.

Key terms: hardware, parallelization, numerical precision, energy per operation, scale, total energy consumption, latency, high-performance memory, power draw, throughput

4.6.4 Cost as a research constraint

This section explains how the rising cost of training and deploying large models determines which research approaches are tested, scaled, and ultimately pursued.

Key terms: architecture, scale, training strategy, stable training dynamics, existing hardware, smaller models, frontier scale, planning cycles, careful validation, rapid iteration

4.7 When scale becomes a gatekeeper

This section introduces how the compute, data, and infrastructure demands of large language models turn scale into a gatekeeper of participation and influence.

Key terms: compute, data, infrastructure, large language models, scale, gatekeeping, participation, influence, access, large language model development

4.7.1 The narrowing field

This section explains how frontier-scale language-model development becomes concentrated among organizations able to execute large, sustained training efforts.

Key terms: frontier scale, execution barrier, full-scale training, sustained operational commitment, end-to-end model development, frontier training, high-quality public sources, proprietary data, specialized data, newly generated data

4.7.2 Infrastructure as power

This section explains how concentrated compute infrastructure, rather than algorithms alone, determines who can build and experiment with frontier-scale language models.

Key terms: frontier scale, concentrated compute, GPUs, TPUs, high-bandwidth networking, reliable energy supply, feasibility, infrastructure as power, technical leadership, architectural ideas

4.7.3 Centralization and dependency

This section explains how centralized LLM platforms make adoption easier while creating organizational dependency on external providers.

Key terms: platforms, APIs, managed deployments, underlying infrastructure, dependency, centralization, availability, pricing, acceptable use, structural reliance

4.7.4 Who gets to know

This section explains how the concentration of frontier-model resources creates an epistemic divide over who can investigate, verify, and generate knowledge about large language models.

Key terms: centralization, frontier systems, controlled retraining, large-scale experimentation, internal states, epistemic divide, limited interfaces, partial disclosures, inference from outputs, full reproducibility

4.7.5 An uneven field

This section explains how the accumulating resource demands of large language models create a persistent imbalance between widespread use and concentrated control over model creation.

Key terms: gatekeeping effects of scale, accumulation, computation, infrastructure, coordination, concentration, divergence between use and authorship, adaptation, integration, application

4.8 Summary

This summary connects Transformer design, scaling and adaptation, multimodality, and the economic consequences that shape who can build and use large language models.

Where this is used later

Concepts explained in this chapter

← Chapter 3: Artificial neural networks Chapter 5: Back to humans →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.