liveBook · Manning

Chapter 1: Introduction to Triton

What you will learn in chapter 1 of GPU Programming with Triton

The introduction explains why growing AI compute demands make GPUs important, why Triton fills a gap between ML frameworks and CUDA, and how combining operations in one kernel can reduce overhead.

After this chapter you will be able to

Section by section

Introduction

The introduction explains why growing AI compute demands make GPUs important, why Triton fills a gap between ML frameworks and CUDA, and how combining operations in one kernel can reduce overhead.

Key terms: GPU, compute demands, compute budgets, Triton, GPU kernels, CUDA, low-level performance, GPU kernel, eager execution, kernel launch

1.1 The philosophy and foundations of Triton

This section explains why a correct PyTorch prototype may still need GPU-level optimization and how Triton makes that optimization accessible from Python.

Key terms: profiler, per-operation breakdown, GPU underutilized, kernels, Triton, Python-first, GPU kernels, compiler, CUDA

1.1.1 What is Triton?

This section defines Triton as a Python-embedded GPU language and compiler, then explains why its specialized, tile-based model supports productivity, performance, and portability.

Key terms: embedded domain-specific language, compiler, @triton.jit, runtime, domain-specific language, embedded DSL, blocked tensor operations, block-based language, tile, block

1.1.2 When to use Triton

This section explains when writing a Triton kernel is worthwhile, compared with relying on torch.compile() or choosing lower-level GPU programming.

Key terms: Triton, CUDA, hand-tuned CUDA, PyTorch pipeline, torch.compile(), TorchInductor, Triton kernels, custom activation, unusual normalization layer

1.2 How Triton fits into the GPU ecosystem

Defines GPU programming in this book as using compute APIs and briefly situates Triton among other interfaces in a CUDA-dominated ecosystem.

Key terms: compute APIs, general-purpose computation, host program, CUDA, Python-native eDSLs, toolchain investment

1.2.1 Levels of abstraction in GPU programming

This section places Triton on a GPU programming abstraction spectrum and explains the trade-off between ease of use and control over execution.

Key terms: spectrum of abstraction, control, productivity, high-level libraries, CUDA, PTX, ptxas, SASS, virtual assembly language, Triton

1.2.2 The Python GPU DSL landscape

This section explains why Python GPU DSLs make different tradeoffs between programmer control and compiler automation, placing Helion, Triton, and Gluon along that spectrum.

Key terms: Python DSLs, memory layout, hardware scheduling, compiler, Helion, Triton, Gluon, autotuner, blocked kernels, tile shapes

1.2.3 Triton in the wild: libraries and use cases

The section shows how Triton is used in mainstream ML frameworks, LLM training libraries, and inference engines, and distinguishes it from NVIDIA’s similarly named inference server.

Key terms: PyTorch 2.x, torch.compile, TorchInductor, Triton kernels, Unsloth, Liger-Kernel, attention, backpropagation, vLLM, SGLang

1.3 Thinking in parallel : GPU execution model

The section explains why understanding GPU hardware matters before writing performance-focused Triton code.

Key terms: Triton-powered libraries, GPU, underlying hardware

1.3.1 Concurrency vs parallelism

The section distinguishes concurrency from parallelism, explains why tensor workloads need parallel execution, and clarifies a basic difference in GPU terminology across vendors.

Key terms: concurrency, parallelism, independent tasks, waiting, tensor operations, arithmetic, Streaming Multiprocessors, SMs, Compute Units, CUs

1.3.2 Why GPUs are designed differently

The section explains why GPU hardware favors large amounts of independent work, when CPU execution can still be preferable, and how the CPU and GPU cooperate in a GPU-accelerated program.

Key terms: embarrassingly parallel, element-wise tensor operations, data-parallel, latency-optimized, throughput-optimized, caches, control logic, ALUs, kernel launch, SIMD

1.3.3 The GPU execution model

This section explains how Triton’s tile-level kernels map onto GPU threads, warps, blocks, grids, and streaming multiprocessors during execution.

Key terms: program instance, block, tile, kernel, grid, thread, warp, streaming multiprocessor, GPU thread, CPU thread

1.4 How GPUs handle data

Explains why moving data can limit GPU performance, how GPUs hide memory latency and organize storage, and how a Triton kernel reaches execution.

Key terms: distance problem, latency hiding, clock cycles, registers, shared memory, L1 cache, L2 cache, global memory, memory hierarchy, SRAM

1.5 Summary

1.6 Exercises

Where this is used later

Concepts explained in this chapter

Chapter 2: Write your first kernel in Triton →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.