PyTorch
PyTorch is an open-source machine-learning framework that provides tensor computation, automatic differentiation, and tools for developing and training neural networks.
Inside Deep Learning: Math, Algorithms, Models
A Python deep learning library that provides tensor computation, automatic differentiation, and data abstractions for neural network training.
- How it is used
- Supports neural-network training by representing data and parameters as tensors and supplying gradients for loss minimization.
- How deep it goes
- Works through tensor operations, GPU placement, differentiation, and optimizer loops, without descending into framework internals.
- Examples
- Uses tensor shapes, CPU–GPU matrix multiplication, quadratic minimization, an SGD loop, and a PCA autoencoder.
- What you can do after
- Apply it via PyTorch to create tensors, place computations on a device, calculate gradients, and update parameters with SGD.
Read how Inside Deep Learning: Math, Algorithms, Models explains PyTorch →
Deep Learning with R, Third Edition
PyTorch is an open-source machine-learning library providing tensor computation and automatic differentiation capabilities.
- How it is used
- Uses PyTorch for tensor operations, gradient-based training, and as a Keras backend.
- How deep it goes
- Works from tensors and automatic differentiation through manual and optimizer-based training steps, including compilation.
- Examples
- Demonstrates a squared-tensor gradient, a linear classifier, and a custom Keras training step.
- What you can do after
- Apply PyTorch via R and reticulate to compute gradients and implement a classifier training step.
Read how Deep Learning with R, Third Edition explains PyTorch →
Deep Learning with PyTorch, Second Edition
The PyTorch core Python library namespace that exposes tensor operations and functions such as mathematical elementwise operations.
- How it is used
- Supports the workflow from tensor operations and model definition through training and deployment.
- How deep it goes
- Introduces tensor creation and indexing, outlines autograd and accelerated operations, and sets exercises for mathematical functions.
- Examples
- Creates and edits a tensor of ones and proposes exercises with views, storage, cosine, and square root.
- What you can do after
- Apply PyTorch to construct tensors and build, train, and deploy deep learning models.
Read how Deep Learning with PyTorch, Second Edition explains PyTorch →
Build AI Drug Discovery Pipelines
PyTorch is an open-source machine-learning framework for building and training neural networks with tensor computation and automatic differentiation.
- How it is used
- Uses PyTorch to construct, train, and evaluate neural networks for ligand-based virtual screening.
- How deep it goes
- Works through tensors and a molecular prediction workflow, then addresses neural-network mechanisms without attempting comprehensive PyTorch coverage.
- Examples
- Demonstrates tensor operations and an EGFR compound-activity model with test-set error metrics.
- What you can do after
- Apply it via PyTorch to prepare molecular data, train a neural network, evaluate predictions, and save model parameters.
Read how Build AI Drug Discovery Pipelines explains PyTorch →
Natural Language Processing in Action, Second Edition
PyTorch is an open-source machine-learning framework for building and training neural-network models with tensor computation and automatic differentiation.
- How it is used
- Uses PyTorch to make neural-network training explicit and implement convolution over token features.
- How deep it goes
- Works through model definition, loss, optimization, training and evaluation, then constructs a convolution layer with manually loaded weights.
- Examples
- Uses character n-grams to classify names and part-of-speech vectors to locate an adverb–verb–noun pattern.
- What you can do after
- Apply it via PyTorch to define and train a single-neuron classifier and run a manually weighted convolution.
Read how Natural Language Processing in Action, Second Edition explains PyTorch →
Deep Learning with Python, Third Edition
A Python machine-learning library used here as one of the frameworks whose code can run element-wise operations on GPUs; only PyTorch is actually a library entry in the glossary, but the section groups it with JAX and TensorFlow as frameworks.
- How it is used
- Uses PyTorch for tensor computations and neural-network training, including as a Keras backend.
- How deep it goes
- Works through tensors, gradients, modules, optimizers, and compilation, then builds a linear classifier with manual and optimizer-based training steps.
- Examples
- Uses tensor creation and gradient calculations, a linear classifier, and top-class gradients for ConvNet interpretation.
- What you can do after
- Apply it via PyTorch to define a model, compute gradients, and perform optimizer-based training steps.
Read how Deep Learning with Python, Third Edition explains PyTorch →
Learn Generative AI with PyTorch
A deep learning library used here for building and training neural networks, especially with dynamic computation graphs and GPU acceleration.
- How it is used
- Uses PyTorch as the Python framework for deep learning and generative modeling.
- How deep it goes
- Explains dynamic computational graphs and GPU training, then outlines an end-to-end classification workflow.
- Examples
- Uses labeled grayscale clothing images to illustrate a 10-class network with dense layers, cross-entropy loss, and Adam.
- What you can do after
- Apply PyTorch to build and train neural networks for binary and multicategory classification.
Read how Learn Generative AI with PyTorch explains PyTorch →
Math and Architectures of Deep Learning
An open-source machine learning library used here to provide the chapter's executable code examples and automatic gradient computation.
- How it is used
- Provides executable counterparts for mathematical operations, statistical models, neural-network training, and convolution in machine-learning workflows.
- How deep it goes
- Demonstrates practical APIs and tensor shapes through worked code, while stopping short of library internals and learned-kernel theory.
- Examples
- Demonstrates matrix diagonalization, PCA and LSA, classifier training, custom convolution kernels, and dimensionality reduction.
- What you can do after
- Apply via PyTorch to implement mathematical transformations, construct training pipelines, optimize models, and compute projections or convolutions.
Read how Math and Architectures of Deep Learning explains PyTorch →
CUDA for LLMs
PyTorch is an open-source machine learning framework with tensor computation and automatic differentiation capabilities.
- How it is used
- Provides a high-level, optimized reference implementation for training neural networks and verifying lower-level versions.
- How deep it goes
- Defines an MNIST MLP, initialization, GPU data setup, and timed training, while deferring implementation internals to CUDA versions.
- Examples
- Uses MNIST classification, a CUDA-resident MLP, manual He initialization, and timed training results.
- What you can do after
- Build a PyTorch MNIST MLP and benchmark its training stages as a baseline for CUDA implementations.
Optimization Algorithms: AI techniques for design, planning, and control problems
PyTorch is an open-source machine-learning library that provides tensor computation and automatic differentiation for building and training neural networks.
- How it is used
- Builds and trains actor and critic neural networks for CartPole reinforcement learning.
- How deep it goes
- Demonstrates network definitions, tensor-based action selection, advantage calculation, optimization, tracking, and visualization.
- Examples
- Uses CartPole actors and critics, episode-reward plots, and Stable-Baselines3 training and evaluation scripts.
- What you can do after
- Builds and trains a reinforcement-learning agent with custom PyTorch networks or Stable-Baselines3.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
PyTorch is an open-source machine learning framework for tensor computation and neural-network development.
- How it is used
- Supports neural-network development and tensor computation for deep learning, including tabular prediction workflows.
- How deep it goes
- Compares PyTorch's stack role and toolkit options, but defers detailed layer definitions and implementation mechanics.
- Examples
- Uses Airbnb NYC price classification and compares framework stacks for tabular deep learning.
- What you can do after
- Evaluate/understand PyTorch ecosystem choices for building deep-learning solutions to tabular-data problems.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains PyTorch →
Fight Fraud with Machine Learning
PyTorch is an open-source machine-learning framework providing tensor computation and automatic differentiation for building and training neural networks.
- How it is used
- Builds deep neural-network models for fraud-detection workflows alongside TensorFlow and classical machine-learning libraries.
- How deep it goes
- Defines class-based architectures, layers, and forward execution, but supplied evidence does not show complete training or evaluation code.
- Examples
- Uses an eight-input, three-hidden-layer sigmoid network represented through Linear and Sigmoid layers.
- What you can do after
- Build it from scratch as a PyTorch neural-network class using nn.Module, ModuleList, Linear, Sigmoid, and forward.
Read how Fight Fraud with Machine Learning explains PyTorch →
Build a Large Language Model (From Scratch)
The deep learning library used here to normalize attention scores along a specified tensor dimension with softmax.
- How it is used
- Uses PyTorch to implement LLMs and support their pretraining or fine-tuning.
- How deep it goes
- Identifies PyTorch as the implementation library but defers comprehensive library instruction elsewhere.
- Examples
- Names BloombergGPT and medical question-answering models as examples of specialized custom LLMs.
- What you can do after
- Apply PyTorch to build LLMs and prepare them for pretraining or fine-tuning.
Read how Build a Large Language Model (From Scratch) explains PyTorch →
Build a Reasoning Model (From Scratch)
A deep learning library used here to represent tensors and define optimizers that minimize losses.
- How it is used
- Uses PyTorch to build and train neural networks within a pretrained-LLM text-generation workflow.
- How deep it goes
- Defines PyTorch and explains installation choices, but stops short of neural-network or tensor implementation details.
- Examples
- Uses requirements files, direct torch installation commands, and uv-based environment workflows.
- What you can do after
- Operate/configure PyTorch by selecting a compatible build and installing it with pip or uv.
Read how Build a Reasoning Model (From Scratch) explains PyTorch →
Also covered in
- Build a Text-to-Image Generator (from Scratch)
- Bayesian Optimization in Action
- Hugging Face in Action
- Data Without Labels
- Machine Learning System Design
- Julia as a Second Language
- Causal AI
- Grokking GPU ProgrammingIn early access (MEAP)
- Evolutionary Deep Learning: Genetic algorithms and neural networks
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- PythonPython is a high-level, general-purpose, dynamically typed programming language with readable syntax and support for multiple programming paradigms.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- backpropagationBackpropagation is reverse-mode differentiation that propagates derivatives backward through a computational graph to compute gradients of an output or loss with respect to inputs and parameters.
Related concepts
- CUDAuses · CUDA is NVIDIA’s parallel computing platform and programming model for performing general-purpose computations on NVIDIA GPUs.
- Captumused by · Captum is an open-source library for interpreting machine-learning models, including through attribution methods.
- cuBLASuses · cuBLAS is NVIDIA’s GPU-accelerated library of basic linear algebra operations for CUDA applications.
- PyTorch Geometricused by · PyTorch Geometric is an open-source Python library that extends PyTorch with data structures and operators for graph and geometric machine learning.
- fastaiused by · fastai is a high-level deep learning library built on PyTorch that provides tools and abstractions for training and deploying models.
- reference implementationused by · A reference implementation is a clear, correctness-oriented implementation that demonstrates intended behavior and provides a basis for validating other implementations.
Questions about PyTorch
What is PyTorch?
PyTorch is an open-source machine-learning framework that provides tensor computation, automatic differentiation, and tools for developing and training neural networks.
Which Manning books explain PyTorch?
Inside Deep Learning: Math, Algorithms, Models, Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Build AI Drug Discovery Pipelines, Natural Language Processing in Action, Second Edition, Deep Learning with Python, Third Edition, Learn Generative AI with PyTorch, Math and Architectures of Deep Learning and 6 more.
What should I know before learning PyTorch?
Manning books that teach PyTorch usually introduce language model, Python, objective function and backpropagation first.
What is PyTorch also called?
PyTorch is also called torch.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.













