liveBook · Manning

What you will learn in GPU Programming with Triton

Accelerate AI training and inference

by Harshwardhan Fartale · Manning Early Access Program (MEAP)

Chapter by chapter

Chapter 1: Introduction to Triton

The introduction explains why growing AI compute demands make GPUs important, why Triton fills a gap between ML frameworks and CUDA, and how combining operations in one kernel can reduce overhead.

1.1 The philosophy and foundations of Triton · 1.1.1 What is Triton? · 1.1.2 When to use Triton · 1.2 How Triton fits into the GPU ecosystem · 1.2.1 Levels of abstraction in GPU programming · 1.2.2 The Python GPU DSL landscape · 1.2.3 Triton in the wild: libraries and use cases · 1.3 Thinking in parallel : GPU execution model · 1.3.1 Concurrency vs parallelism · 1.3.2 Why GPUs are designed differently · 1.3.3 The GPU execution model · 1.4 How GPUs handle data · 1.5 Summary · 1.6 Exercises

What you learn in chapter 1 →

Chapter 2: Write your first kernel in Triton

Introduces the chapter’s plan to build, examine, launch, and benchmark a vector-addition kernel in Triton.

2.1 The Triton programming model · 2.1.1 Thinking in blocks, not threads · 2.1.2 The Single Program, Multiple Data model · 2.2 A complete Triton kernel: vector addition end to end · 2.2.1 Imports and the JIT decorator · 2.2.2 Function signature and pointers · 2.2.3 Getting a unique program ID · 2.2.4 Calculating the data chunk · 2.2.5 Computing the element offsets · 2.2.6 Handling edges with a mask · 2.2.7 Loading data and performing the computation · 2.2.8 The launch grid · 2.2.9 Kernel execution flow · 2.2.10 Benchmarking the kernel · 2.3 A high-level language with a low-level compiler · 2.3.1 The compilation funnel (advanced) · 2.3.2 Practical inspection of compilation stages · 2.4 Summary

What you learn in chapter 2 →

Chapter 3: Fused kernels and parallel patterns

The introduction recalls the independent per-program pattern of earlier Triton kernels and frames the move toward more complex workloads.

3.1 The memory wall and kernel fusion · 3.1.1 Embarrassingly parallel tasks: the baseline GPU pattern · 3.1.2 SiLU as an embarrassingly parallel kernel · 3.1.3 Arithmetic intensity and the memory wall · 3.1.4 Compute-bound vs. memory-bound · 3.1.5 Roofline model · 3.2 The GPU Memory System · 3.2.1 Registers hold what matters right now · 3.2.2 Shared memory enables cooperation · 3.2.3 Caches provide transparent speedup · 3.2.4 Global memory trades capacity for latency · 3.3 Kernel fusion: the solution · 3.3.1 Case Study: SwiGLU · 3.4 Control flow: masking over branching · 3.4.1 Warp divergence · 3.4.2 Predicated execution with tl.where · 3.5 Advanced indexing: coordinate mapping (RoPE) · 3.5.1 Positional encoding in transformers · 3.5.2 The memory layout challenge · 3.5.3 Building the kernel · 3.5.4 Why coordinate mapping matters · 3.6 Exercises · 3.6.1 Know thy GPU · 3.6.2 Value clipping · 3.6.3 Full elementwise RoPE · 3.7 Summary

What you learn in chapter 3 →

Chapter 4: Reduction patterns

Introduces reductions as GPU collective operations that turn distributed values into a shared statistic through coordinated communication.

4.1 The mechanics of reduction · 4.1.1 Floating-point associativity and numerical concerns · 4.1.2 Triton’s primitives for reduction · 4.1.3 Atomic operations · 4.1.4 RMS normalization in Triton · 4.2 Numerical stability and chained reductions (Softmax) · 4.2.1 The "exploding exponential" problem · 4.2.2 Case study: softmax in Triton · 4.2.3 Fused softmax kernel · 4.3 Summary

What you learn in chapter 4 →

Chapter 5: Higher-dimensional kernels

The chapter introduction explains why matrix multiplication matters in deep learning and why matrix kernels require thinking about both logical dimensions and linear memory.

5.1 Row-major and column-major ordering · 5.2 Strides: the language of tensor layout · 5.2.1 A concrete example · 5.2.2 The same memory, two tensors · 5.2.3 Stride interpretation and special values · 5.2.4 Memory coalescing: why access patterns affect performance · 5.3 Matrix addition · 5.3.1 Building the first 2D kernel · 5.3.2 2D program ID decomposition and tile indexing · 5.3.3 Boundary masking for non-tile-aligned shapes · 5.3.4 The complete matrix-addition kernel · 5.4 Matrix transpose · 5.4.1 Naive transpose kernel · 5.4.2 Why uncoalesced writes hurt · 5.4.3 Staging the transpose through shared memory · 5.4.4 Bank conflicts and the one-column padding fix · 5.5 Matrix multiplication · 5.5.1 Extending matrix multiplication with scaling and accumulation · 5.5.2 Naive matrix multiplication kernel · 5.5.3 Arithmetic intensity and the roofline view · 5.5.4 Reusing values across output elements · 5.5.5 The first tiled kernel in Triton · 5.6 The Triton primitives behind the tiled kernel · 5.6.1 The tl.dot block contraction · 5.6.2 Describing tiles with tl.make_block_ptr · 5.6.3 Advancing tiles with tl.advance · 5.7 Why tiled matrix multiplication uses Tensor Cores · 5.8 From aligned tiles to an optimized matrix multiplication · 5.8.1 Handling incomplete edge tiles · 5.8.2 Reordering output tiles to reuse L2 data · 5.8.3 Letting the autotuner choose a launch configuration · 5.8.4 Adding compiler hints carefully · 5.8.5 Final optimized matrix multiplication kernel · 5.8.6 Checking correctness across shapes and dtypes · 5.9 Benchmarking the matrix multiplication progression · 5.10 Summary

What you learn in chapter 5 →

Concepts explained in this book

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.