Chapter 1: Generative AI: Why large language models have captured attention

Introduction

The chapter introduction compares the excitement around LLMs to the arrival of the printing press and previews questions the chapter will address.

• Printing press and LLM analogy: Explain why the introduction compares LLMs with the printing press: both prompted excitement about new ways to spread information, and LLMs can rapidly generate text and accelerate communication.
  Why it matters: The analogy frames why LLMs have attracted attention before the chapter examines them in detail.
• Chapter questions ahead: State the three questions the introduction says the chapter will address: what LLMs can and cannot do, when to deploy your own LLM, and which common claims about LLMs are myths.
  Why it matters: These questions orient the reader to the chapter without assuming the introduction has already answered them.

Key terms: printing press, large language models, accelerate communication, deploy your own large language models, myths

1.1 Large language models accelerating communication

This section explains why LLMs attract attention as communication tools, where they are useful or unsuitable, and why turning their promise into products takes more than a capable model.

• Communication as the central value: Explain that LLMs are most useful for improving or automating language-based work, especially routine communication that takes time away from more valuable work. Describe examples such as assistants, coding copilots, tutoring, or talking to data.
  Why it matters: The communication task is the starting point for deciding whether an LLM could help.
• Task fit and operating constraints: Distinguish good communication-oriented uses from latency-sensitive or simple tasks where an LLM may be too slow, complex, and costly. Explain that problems better handled by math or other algorithms, such as forecasting, are also poor fits.
  Why it matters: A model's ability to produce an answer does not make it the right tool for the job.
• High stakes and human accountability: Explain why LLM mistakes and potentially variable answers make unsupervised decisions risky in areas such as investments, taxes, and medical advice. Distinguish using an LLM to assist a responsible practitioner from letting it make consequential decisions on its own.
  Why it matters: In a high-stakes setting, people remain responsible for decisions made with an LLM's output.
• Promise versus production products: Explain the gap between impressive demonstrations and useful products: an LLM must work coherently for users, and production brings pitfalls unlike traditional software. Describe the need for a team with technical and user-experience skills, whether the model is deployed in-house or accessed through a vendor API.
  Why it matters: Turning a promising model into a usable service requires decisions and work beyond choosing the model.

Key terms: communication, AI assistants, programming assistants, talk-to-your-data, latency-sensitive workloads, autocomplete, forecasting, outlier prediction, temperature, oversight

1.2 Navigating the build-and-buy decision with LLMs

This section argues that LLM ideas create value only when models are put into production, then introduces the choice between buying a provider’s offering and building an alternative.

• Production over promising ideas: Explain why the section emphasizes getting LLMs into production: promising ideas alone do not do work, while deployed models can be readily available for actual use.
  Why it matters: The book focuses on turning LLM potential into something people can actually use.
• Build-and-buy decision framing: Describe why buying an LLM offering is tempting when production deployment is hard, and state that the section will examine both when provider offerings are a good choice and where they fall short.
  Why it matters: The appeal of a shortcut is easier to assess when you also consider its limits.

Key terms: production, execution, build-and-buy decision, deploying LLMs into production, OpenAI, Google

1.2.1 Buying: The beaten path

This section explains why buying LLM access can be a practical way to prototype quickly, use specialized models and provider support, and focus on the application around the model.

• API prototyping and validation: Explain how an LLM API lets a team build and launch a prototype quickly and cheaply, then test whether the model works in its domain before investing heavily.
  Why it matters: A quick prototype can reveal a poor fit before the team commits substantial resources.
• Specialized model data advantage: Justify buying access when a provider has a strong domain-specific model backed by specialized data that would be costly for your team to curate.
  Why it matters: The quality of domain-specific data can matter more than a team's desire to build its own model.
• Provider expertise and safeguards: Describe how buying model access can also provide the provider's expertise, support, and surrounding systems, including safety filtering and controls already developed for edge cases.
  Why it matters: A provider may have already done work that would otherwise fall to your team.
• Application experience beyond models: Explain that obtaining an LLM does not finish a production application: a team still needs to build the application and its user experience, such as presenting generated tokens as if they are being typed.
  Why it matters: Model access leaves important product and user-experience work to be done.

Key terms: API, prototype, fail fast, domain-specific data, finetune, curating data, expertise and support, filtering and controls, edge cases, production

1.2.2 Building: The path less traveled

This section explains why an organization might build and deploy its own LLM despite the ease of buying an API service, then frames build and buy as choices that can serve different project phases.

• Control over model behavior: Explain how owning an LLM gives an organization more control over model behavior, training, deployment, and vendor-dependent decisions. Use the AI Dungeon example to show what can happen when a provider's requirements conflict with a product's needs.
  Why it matters: When an LLM is central to a product, limited control can leave a team unable to fix problems at their source.
• Task-specific competitive advantage: Explain how training an open source model on an organization's own data can produce a model that excels at its particular tasks without requiring general-purpose intelligence. Relate that specialization to competitive advantage and keeping valuable data from potential competitors.
  Why it matters: A model that is especially good at the work a business needs may be more valuable than a broadly capable model.
• Local integration and reliability: Describe why deploying an LLM within an organization's system, including potentially on an edge device, can suit applications where internet access, API latency, or provider availability is a problem.
  Why it matters: Where and how a model runs can determine whether an application works reliably for its users.
• Build versus buy costs: Explain why the apparent low price and ease of a bought service do not settle the cost decision. Contrast potentially expensive self-hosted inference with inexpensive local proofs of concept or more efficient training approaches, and mention exposure to vendor price changes.
  Why it matters: The relevant costs depend on the workload and project stage, not just a service's advertised price.
• Sensitive data and third parties: Explain why sending sensitive manuals, source code, customer information, or trade secrets to an external LLM service can create privacy and security concerns. Distinguish those concerns from a blanket ban on third-party processing by noting that provider policies can change and trusted, contracted arrangements matter.
  Why it matters: A useful LLM application can still be unacceptable if its data handling exposes information the organization must protect.
• Phase-dependent build and buy: Describe how the section weighs the build-or-buy decision rather than prescribing one choice for every project. Explain the proposed combination: buy for fast, easy research or prototyping, while considering building for production needs such as privacy, control, and flexibility.
  Why it matters: Choosing differently at different stages can capture the speed of buying without giving up capabilities a production system may need.

Key terms: control, AI Dungeon, Latitude, finetuning, competitive edge, open source LLMs, BERT, SlovenBERTcina, trade secrets, integrate anywhere

1.2.3 A word of warning: Embrace the future now

Uses Borders’ failure to adapt to e-commerce to warn that relying entirely on outside LLM providers may leave companies without expertise needed for future technologies.

• Borders’ lost technology advantage: Trace how Borders first gained an advantage through inventory technology, then lost ground after outsourcing online sales instead of developing its own e-commerce capabilities.
  Why it matters: The Borders story shows how a company can benefit from one technological shift yet be unprepared for the next.
• In-house expertise for future shifts: Explain why the section urges companies to gain hands-on LLM deployment expertise now: later technologies may build on LLMs, just as e-commerce built on the internet, and relying only on an API may leave a company unable to adapt.
  Why it matters: The payoff from learning to deploy LLMs may be the ability to respond to a future shift, not just an immediate business gain.
• Accessibility beyond large corporations: State that the section rejects the idea that only corporations with millions of dollars and thousands of GPUs can work with LLMs; companies of any size can get started.
  Why it matters: The warning to start building expertise applies to smaller companies too.

Key terms: Borders, inventory management system, e-commerce, Amazon, Kindle, Kobo, in-house expertise, LLMs, deployment, API

1.3 Debunking myths

The section challenges myths about who can work with LLMs, what it takes to build with them, and why a compelling demo is not yet a reliable product.

• Starting without training from scratch: Explain why solving a problem with an LLM usually does not require training one from scratch: open source base models provide starting points, and evolving tools make them easier to run and adapt.
  Why it matters: Choosing an existing model can remove a major barrier without pretending that training a new one is easy.
• Accessible paths into LLM work: Describe why a career at a large AI company or expertise in every contributing field is not a prerequisite for getting involved with LLMs. Give the section’s examples of learning through participation or hands-on projects.
  Why it matters: There are ways to start contributing and learning without first changing careers.
• Demo versus functioning product: Distinguish an impressive LLM prototype from a functioning product that responds reliably to real users at scale. Explain how the section uses Watson’s commercial struggles and ChatGPT’s accessibility to make that distinction.
  Why it matters: A demo that works once does not establish that an application will work consistently for its users.
• Reliability and output risks: Explain why LLM applications must address hallucinations and unsafe outputs rather than assuming the model will be consistently correct. Describe the section’s proposed use of context, alignment techniques, training or prompting, and attention to what the model reveals or generates.
  Why it matters: Making a model useful includes managing what it says, not only securing what goes into it.
• Realistic costs of LLM adoption: State the section’s caution that LLMs remain resource-intensive, difficult to understand, and difficult to make behave as intended, even though tools and techniques are reducing the burden. Explain that buying from a vendor may still be the right choice.
  Why it matters: Lower barriers to entry do not remove the need for a realistic build-or-buy decision.

Key terms: open source models, base models, training from scratch, frameworks and libraries, OpenAssistant, linguistics, mathematics, computer science, hands-on examples, prototype

1.4 Summary

This summary recaps why language makes LLMs broadly useful, when they are a poor fit, and the tradeoffs between buying and building them.