chapter one

1 Peeking inside the black box

 

In this chapter

  • You will learn what’s needed to develop useful GenAI applications.
  • You will get an intuition of how Large Language Models work.

While it feels natural to use tools like ChatGPT or Copilot these days, you realize what a strange animal a GenAI (short for Generative Artificial Intelligence) application can be when you try to build one yourself.

How can AI systems communicate in fluent, human-like language? Does ChatGPT really think like we do, or is something else at play? In this book, we’ll explore what makes GenAI so unique, mysterious, and fascinating.

Before diving deeper into what makes GenAI applications such an exciting new species, it’s worth taking a moment to clarify the approach of this book and what you should expect from here.

Our journey ahead

The primary goal of this book is to help you understand how GenAI applications are built. Instead of going through mathematical proofs or model architecture internals, we focus on what truly matters when you want to design real-world apps: the main building blocks, how they fit together, the problems they solve, and their limitations. By the end of the book, no matter which tools or frameworks you choose to use, you’ll have a clear mental model for designing and reasoning about GenAI applications.

The uniqueness of GenAI programming

Dealing with LLMs

Building GenAI applications that extend LLMs

Overcoming the “stateless” barrier

Providing fresh knowledge in prompts

How this book helps you build real-world GenAI applications

Inside the LLM

Learning how to speak

Where everything starts: Sentence completions

Understanding GPT: How a real LLM works

Embeddings

Context calculation

Combination process

Pre-training the model

Keeping the knowledge up to date

Summary