CUDA
CUDA is NVIDIA’s parallel computing platform and programming model for performing general-purpose computations on NVIDIA GPUs.
GPU Programming with Triton
CUDA is NVIDIA's parallel computing platform and programming model, including the CUDA C++ language extensions for GPU programming.
- How it is used
- Positions CUDA as the low-level control point against which framework and Triton abstractions are compared.
- How deep it goes
- Explains execution terminology and control over memory, work division, and synchronization, without walking through a CUDA kernel.
- Examples
- Contrasts torch.matmul with a CUDA matrix multiplication implementation and uses tensor elements and pixels to illustrate thread work.
- What you can do after
- Enables evaluation of CUDA’s control and complexity against framework and Triton alternatives.
Starting Data Analytics with Generative AI and Python
CUDA is NVIDIA's parallel-computing platform and programming interface for executing general-purpose computations on compatible GPUs.
- How it is used
- Accelerates suitable numerical workloads by directing repeated vector or matrix operations to an NVIDIA GPU.
- How deep it goes
- Demonstrates a CuPy rewrite of bootstrap computation and notes that benefits depend on workload size and GPU hardware.
- Examples
- Uses a CUDA bootstrap confidence-interval function, CuPy installation, and execution-time comparisons against optimized CPU code.
- What you can do after
- Apply CUDA through CuPy to adapt compatible NumPy-style numerical functions for GPU execution and return usable results.
Read how Starting Data Analytics with Generative AI and Python explains CUDA →
Fast Python: High performance techniques for large datasets
- How it is used
- Deploys NVIDIA CUDA APIs to perform GPU-based data processing, including array operations and Mandelbrot generation.
- How deep it goes
- Builds kernels and launch configurations with bounds checks, but defers advanced shared-state algorithms and detailed C instruction.
- Examples
- Demonstrates doubling one- and two-dimensional arrays and generating a Mandelbrot set with CUDA C.
- What you can do after
- Build GPU array kernels with Numba or CUDA C, configure their thread launches, and compare measured execution times.
Read how Fast Python: High performance techniques for large datasets explains CUDA →
Grokking GPU Programming
CUDA is NVIDIA's parallel computing platform and programming model for executing general-purpose computations on GPUs.
- How it is used
- Enables general-purpose GPU computation and develops parallel-programming understanding that carries across frameworks.
- How deep it goes
- Defines CUDA and outlines kernels, launches, thread indexing, and data transfers, while only previewing detailed execution mechanics.
- Examples
- Uses exam grading, matrix math, image processing, AI training, and an image pipeline to illustrate parallel work and transfer costs.
- What you can do after
- Build it from scratch by writing and launching a CUDA kernel, identifying threads, and transferring data between CPU and GPU.
Hugging Face in Action
CUDA is NVIDIA's parallel computing platform and programming model for executing general-purpose computations on GPUs.
- How it is used
- Accelerates PyTorch-based training, inference, and tensor computation on NVIDIA GPUs.
- How deep it goes
- Defines CUDA and demonstrates installation, availability checks, device selection, and multi-GPU model parallelism without explaining CUDA internals.
- Examples
- Demonstrates CUDA installation, torch diagnostics, GPUtil inspection, sentiment classification, and pretrained sequence-classification loading.
- What you can do after
- Apply it via PyTorch and Transformers to detect GPUs, select CUDA execution, inspect hardware, and accelerate model workloads.
Inside Deep Learning: Math, Algorithms, Models
NVIDIA's GPU computing software stack used here as the compute interface for running PyTorch operations on a GPU.
- How it is used
- Uses CUDA as the compute interface for running PyTorch tensor operations on an NVIDIA GPU.
- How deep it goes
- Demonstrates device selection, tensor transfers, timing, device-mismatch errors, and moving data back for NumPy conversion.
- Examples
- Compares CPU and GPU matrix multiplication and shows a mixed-device error.
- What you can do after
- Operate PyTorch tensors on a CUDA device, compare execution time, and recognize device-mismatch errors.
Read how Inside Deep Learning: Math, Algorithms, Models explains CUDA →
CUDA for LLMs
CUDA is NVIDIA's parallel computing platform and programming model for executing general-purpose computations on GPUs.
- How it is used
- Provides direct GPU programming when framework operations are unavailable or too slow for a workload.
- How deep it goes
- Defines host/device roles, separate memories, transfers, kernels, and thread-block-grid indexing; defers profiling details.
- Examples
- Uses PyTorch model transfer, matrix multiplication, and a naive CUDA multilayer perceptron as examples.
- What you can do after
- Build custom CUDA kernels, recognize performance bottlenecks, and optionally integrate low-level GPU code into PyTorch.
Also covered in
What to know first
- neural networkA neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- matrix-matrix multiplicationMatrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
Related concepts
- PyTorchused by · PyTorch is an open-source machine-learning framework that provides tensor computation, automatic differentiation, and tools for developing and training neural networks.
- cuBLAShas part · cuBLAS is NVIDIA’s GPU-accelerated library of basic linear algebra operations for CUDA applications.
- GPUuses · A graphics processing unit (GPU) is a processor designed for highly parallel computation, especially graphics and other data-parallel workloads.
- deep learningused by · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
Questions about CUDA
What is CUDA?
CUDA is NVIDIA’s parallel computing platform and programming model for performing general-purpose computations on NVIDIA GPUs.
Which Manning books explain CUDA?
GPU Programming with Triton, Starting Data Analytics with Generative AI and Python, Fast Python: High performance techniques for large datasets, Grokking GPU Programming, Hugging Face in Action, Inside Deep Learning: Math, Algorithms, Models and CUDA for LLMs.
What should I know before learning CUDA?
Manning books that teach CUDA usually introduce neural network, CPU, language model and matrix-matrix multiplication first.
What are the parts of CUDA?
CUDA includes cuBLAS, CUDA C and cuDNN.
What is CUDA also called?
CUDA is also called Compute Unified Device Architecture.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






