Chapter 1: Introducing deep learning and the PyTorch library
The introduction positions PyTorch as an approachable Python library for building deep learning projects.
After this chapter you will be able to
- PyTorch’s approachable Python design. Explain that PyTorch helps developers build deep learning projects by emphasizing flexibility and letting them express models in idiomatic Python, an approach that attracted early research adopters.
- Learning complex tasks from examples. Explain that deep learning is an AI subcategory in which deep neural networks are trained with instructive examples and large amounts of data to approximate complex functions, rather than being given handcrafted rules for each task.
- Handcrafted and learned features. Distinguish traditional feature engineering from deep learning’s automatic discovery of useful representations, using the handwritten-digit example. Explain that deep learning reduces, but does not eliminate, the need to supply prior knowledge.
- Loss-guided training. Explain what a loss function measures and how training uses it to improve a model, including the aim of performing well on unseen data.
- PyTorch as a starting library. Explain why the authors recommend PyTorch as a first deep learning library while also considering it suitable for professional work.
- Tensor foundation and computation. Describe the basic view of a deep learning system given here and how PyTorch supports expressing and computing it.
- Familiar and expressive programming. Explain how PyTorch makes deep learning programming feel familiar to Python developers while allowing them to express complicated models without undue framework complexity.
- Accelerated computation and optimization. Explain the two capabilities the section identifies as particularly relevant to deep learning: GPU-accelerated computation and support for numerical optimization during training.
- Framework landscape consolidation. Describe the shift from many competing deep learning libraries to a landscape in which PyTorch is prominent in research and PyTorch, TensorFlow, and Hugging Face are widely used in industry.
- Tensors and their operations. Describe tensors and the torch operations on them as PyTorch’s foundation, including where those computations can run.
- Operation history and autograd. Explain what tensors’ operation history lets autograd calculate and why that calculation matters to learning.
- Dataset and DataLoader roles. Distinguish how Dataset bridges custom samples to tensors from how DataLoader prepares batches for the training loop.
- Model training loop components. Trace the section’s basic training loop and identify the roles of torch.nn, a loss function, autograd, and an optimizer.
- Computing needs by workload. Explain why a standard laptop is enough for part 1, while full training runs in part 2 benefit substantially from a GPU without strictly requiring one.
- Cancer dataset storage needs. Describe the download and disk-space needs of the part 2 cancer-detection project, including why training requires more space than the raw download.
- Notebook kernel and persistent state. Explain that a notebook appears in a browser, while a server-side kernel executes its code and returns results. Describe how variables remain available until the kernel is restarted or terminated.
- Cell execution and inline results. Describe how to run a code cell, how later cells can use earlier variables, and where a cell’s final value or plots appear. Explain that notebooks can combine code, results, and Markdown text.
Section by section
Introduction
The introduction positions PyTorch as an approachable Python library for building deep learning projects.
- PyTorch’s approachable Python design: Explain that PyTorch helps developers build deep learning projects by emphasizing flexibility and letting them express models in idiomatic Python, an approach that attracted early research adopters.
Why it matters: This explains why the book uses PyTorch to introduce deep learning practice.
Key terms: PyTorch, flexibility, idiomatic Python
1.1 What is deep learning?
Defines deep learning as training neural networks from examples to perform complex tasks, while distinguishing those capabilities from human thought or self-awareness.
- Learning complex tasks from examples: Explain that deep learning is an AI subcategory in which deep neural networks are trained with instructive examples and large amounts of data to approximate complex functions, rather than being given handcrafted rules for each task.
Why it matters: This is the basic idea behind the models and training examples used throughout the book. - Capability without human thought: Distinguish machines performing tasks associated with human intelligence from machines thinking or being self-aware in a human sense.
Why it matters: It helps you describe what these systems accomplish without assuming they think like people.
Key terms: deep learning, deep neural networks, instructive examples, handcrafted rules, artificial intelligence, self-awareness, approximate complicated, nonlinear processes
1.2 The shift from machine learning to deep learning
This section contrasts handcrafted features with representations learned from data, then explains how a loss function guides training.
- Handcrafted and learned features: Distinguish traditional feature engineering from deep learning’s automatic discovery of useful representations, using the handwritten-digit example. Explain that deep learning reduces, but does not eliminate, the need to supply prior knowledge.
Why it matters: The distinction explains why deep learning changes what practitioners must design by hand. - Loss-guided training: Explain what a loss function measures and how training uses it to improve a model, including the aim of performing well on unseen data.
Why it matters: Loss provides the numerical signal that makes improving a model through training possible.
Key terms: raw features, feature engineering, representations, prior knowledge, training, loss function, criterion, objective function, cost function, reference data
1.3 What to expect
This section explains why PyTorch is an approachable yet professional deep learning library and introduces the tensor as its foundational data structure.
- PyTorch as a starting library: Explain why the authors recommend PyTorch as a first deep learning library while also considering it suitable for professional work.
Why it matters: It explains why learning with PyTorch can be useful beyond introductory exercises. - Tensor foundation and computation: Describe the basic view of a deep learning system given here and how PyTorch supports expressing and computing it.
Why it matters: This gives you a simple starting picture of what PyTorch represents and computes.
Key terms: PyTorch, clear syntax, streamlined API, easy debugging, mathematical function, tensor, multidimensional array, accelerated mathematical operations
1.4 Why PyTorch?
The section explains why PyTorch is a practical choice for deep learning: familiar and expressive Python programming, fast computation with optimization support, and options for deploying models beyond Python.
- Familiar and expressive programming: Explain how PyTorch makes deep learning programming feel familiar to Python developers while allowing them to express complicated models without undue framework complexity.
Why it matters: This explains why PyTorch can be approachable without restricting the models developers can build. - Accelerated computation and optimization: Explain the two capabilities the section identifies as particularly relevant to deep learning: GPU-accelerated computation and support for numerical optimization during training.
Why it matters: These capabilities address the need to train models efficiently on large amounts of data. - Deployment beyond Python: Describe why PyTorch can carry a model from research into production settings where a full Python runtime is impractical.
Why it matters: PyTorch’s usefulness need not end when a flexible research model becomes an application.
Key terms: Tensor, Pythonic, expressivity, GPUs, numerical optimization, training, C++ runtime, inference, mobile devices
1.4.1 The deep learning competitive landscape
This section describes how deep learning tools consolidated around a few major frameworks and distinguishes those frameworks from Hugging Face’s application-oriented tools.
- Framework landscape consolidation: Describe the shift from many competing deep learning libraries to a landscape in which PyTorch is prominent in research and PyTorch, TensorFlow, and Hugging Face are widely used in industry.
Why it matters: This explains where PyTorch fits among the tools a practitioner is likely to encounter. - PyTorch and TensorFlow convergence: Explain why PyTorch and TensorFlow have begun to offer more similar capabilities while remaining different to use.
Why it matters: Similar feature names do not mean the two frameworks feel identical in practice. - Hugging Face application layer: Distinguish Hugging Face’s application-oriented wrapper and model hub from directly authoring models in PyTorch or TensorFlow.
Why it matters: The distinction helps explain why someone might use an existing model through Hugging Face or work directly in a framework.
Key terms: PyTorch, TensorFlow, Hugging Face, consolidation, eager mode, compilation mode, high-level wrapper, model hub, pretrained models
1.5 How PyTorch supports deep learning projects
This section maps PyTorch’s main components onto a deep learning project, from preparing data and training a model to deploying it.
- Python API and execution: Explain how developers usually use PyTorch through Python while performance-critical operations run in C++ or CUDA, and describe what eager execution means.
Why it matters: It connects the Python code you write to the computations PyTorch performs. - Tensors and their operations: Describe tensors and the torch operations on them as PyTorch’s foundation, including where those computations can run.
Why it matters: Tensors are the common representation used throughout the project. - Operation history and autograd: Explain what tensors’ operation history lets autograd calculate and why that calculation matters to learning.
Why it matters: It explains the role of autograd before the book develops the details of training. - Dataset and DataLoader roles: Distinguish how Dataset bridges custom samples to tensors from how DataLoader prepares batches for the training loop.
Why it matters: It separates the problem-specific work of reading samples from the reusable work of supplying batches efficiently. - Model training loop components: Trace the section’s basic training loop and identify the roles of torch.nn, a loss function, autograd, and an optimizer.
Why it matters: It provides the high-level sequence that the later hands-on training chapters fill in. - Trained model deployment paths: Describe why a trained model must be deployed to be useful and name the deployment and scaling options this section identifies.
Why it matters: It places training within the larger project of getting a model into practical use.
Key terms: Python API, C++, CUDA, eager mode, tensors, torch module, CPU, GPU, operation history, automatic differentiation
1.6 Hardware and software requirements
This section explains the computing, software, and storage needed to follow the book, distinguishing ordinary laptop work from more demanding training runs.
- Computing needs by workload: Explain why a standard laptop is enough for part 1, while full training runs in part 2 benefit substantially from a GPU without strictly requiring one.
Why it matters: You can start on the computer you have and plan for faster hardware when training becomes more demanding. - Options for GPU access: Describe the suggested alternative when a local GPU is unavailable and the hardware assumption behind part 2’s default settings.
Why it matters: Lacking a suitable local GPU need not prevent you from attempting the later examples. - Supported systems and installation: State which operating systems PyTorch supports and describe the installation guidance and dependency file provided for readers.
Why it matters: You can choose installation steps appropriate to your operating system rather than assuming the examples require Linux. - Cancer dataset storage needs: Describe the download and disk-space needs of the part 2 cancer-detection project, including why training requires more space than the raw download.
Why it matters: Checking bandwidth and free disk space ahead of time can prevent the larger project from stalling during setup or training.
Key terms: pretrained network, CUDA-capable GPU, parameter updates, 8 GB of RAM, cloud platforms, GPU-enabled Jupyter Notebooks, Linux, macOS, Windows, Anaconda
1.6.1 Using Jupyter Notebooks
This section explains how Jupyter Notebooks run the book’s example code and how to open the notebooks from the code checkout.
- Notebook kernel and persistent state: Explain that a notebook appears in a browser, while a server-side kernel executes its code and returns results. Describe how variables remain available until the kernel is restarted or terminated.
Why it matters: This explains why code in one cell can use variables created earlier. - Cell execution and inline results: Describe how to run a code cell, how later cells can use earlier variables, and where a cell’s final value or plots appear. Explain that notebooks can combine code, results, and Markdown text.
Why it matters: Cells let you experiment with the examples and see results beside the code that produced them. - Notebook server starting location: State where to start the notebook server for the book’s code and what appears after it starts.
Why it matters: Starting in the code checkout makes the book’s notebook files available to open.
Key terms: Jupyter Notebook, kernel, state, cell, Shift-Enter, Markdown-formatted text cells, notebook server, root directory, code checkout
1.7 Exercises
1.8 Summary
Where this is used later
- Learning is just parameter estimation (chapter 5) · Provides the high-level meaning of learning from data before the book develops parameter estimation.
- Artificial neurons (chapter 6) · Introduces deep neural networks before the book explains their components.
- What learning means for a neural network (chapter 6) · Connects learned representations to what learning means for a neural network.
- Less loss is what we want (chapter 5) · Develops the loss measure introduced here.
- Training, validation, and overfitting (chapter 5) · Examines performance on training and unseen validation data.
- Tensors: Multidimensional arrays (chapter 3) · The detailed introduction to tensors builds on their role as PyTorch’s core multidimensional-array data structure.
- Moving tensors to the GPU (chapter 3) · Develops the GPU-computation capability by moving tensors to the GPU.
- PyTorch’s autograd: Backpropagating all things (chapter 5) · Develops PyTorch’s support for optimization during training.
Concepts explained in this chapter
- PyTorchThe PyTorch core Python library namespace that exposes tensor operations and functions such as mathematical elementwise operations.
- language modelA language model with enough scale to generate and answer with fluent text from prompts.
- arrayA container for indexed collections of values.
- trainingThe optimization phase in which a model learns from examples.
- objective functionLoss computed on training data during fitting.
- eager evaluationAn execution model in which operations run immediately rather than being first built into a static graph.
- torch.compile()torch.compile() is a PyTorch API that compiles functions or modules into optimized executable code for execution.
- DataLoaderA PyTorch abstraction that batches and loads samples from datasets.
- automatic differentiationA technique for automatically computing derivatives needed for training neural networks.
- datasetA collection of examples and annotations used for model training or evaluation.
- deploymentThe process of putting a trained model into a real runtime environment where it serves users or applications.
- optimizer stateThe internal state of an optimization algorithm, such as momentum buffers, that can be saved to resume training exactly.
- torch.nnPyTorch’s neural-network module namespace containing classes for layers, containers, and other building blocks.
- convolution kernelA small learned filter window applied over local regions of an input to compute weighted sums.
- fine-tuningThe process of adapting a pretrained model to a new task by continuing training on task-specific data.
- data representationAn internal data encoding produced or consumed by a model.
- Hugging FaceA repository service for discovering, sharing, and exchanging pretrained models and weights.
- ExecuTorchA PyTorch ecosystem library/runtime for deploying models efficiently on constrained devices such as mobile and embedded systems.
- torch.distributedPyTorch’s distributed-training submodule for coordinating computation and communication across processes, devices, or machines.
- classifierA supervised learning task that assigns a text sequence to one of several categories.
- labelA target label restricted to one of two classes, used here in binary classification and metric computation.
- ONNXA vendor-neutral standard and file format for representing neural network computations so models can be executed in different runtimes and devices.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.