CUDA for LLMs cover
CUDA for LLMs cover
welcome to this free extract from
an online version of the Manning book.
to read more
or
about-this-book

about this book

 

CUDA for LLMs teaches CUDA programming through the deep learning workloads where GPU performance matters most. Rather than presenting CUDA as a set of isolated language features, the book builds a practical mental model of how GPU hardware executes neural network kernels, moves data through memory, and exposes performance tradeoffs that high-level frameworks often hide.

The book starts with the foundations: kernels, threads, blocks, memory transfers, indexing, and simple neural network operations. From there it moves into increasingly realistic deep learning kernels: transformer inference, GEMV and GEMM, softmax, layer normalization, top-k, tensor cores, Flash Attention, quantization, distributed communication, and CUTLASS. Each topic is tied back to the same question: what is the GPU doing, and how should that change the code we write?

Throughout the book, you will profile kernels, reason about memory bandwidth and occupancy, compare implementation strategies, and learn when a custom CUDA kernel is worth its complexity. The point is not to replace frameworks for the sake of it. The point is to understand the layer beneath PyTorch, JAX, TensorRT, and other high-level tools well enough to make better performance decisions.

Who should read this book

How this book is organized: A road map

About the code

liveBook discussion forum