liveBook · Manning

Chapter 2: Write your first kernel in Triton

What you will learn in chapter 2 of GPU Programming with Triton

Introduces the chapter’s plan to build, examine, launch, and benchmark a vector-addition kernel in Triton.

After this chapter you will be able to

What it builds on

Section by section

Introduction

Introduces the chapter’s plan to build, examine, launch, and benchmark a vector-addition kernel in Triton.

2.1 The Triton programming model

This section introduces Triton’s division of work between kernel authors and the compiler, and notes that its foundational kernel APIs tend to remain stable across releases.

Key terms: Triton kernel, compiler, parallel computation, memory coalescing, shared memory, instruction scheduling, program IDs, offsets, masks, @triton.jit decorator

2.1.1 Thinking in blocks, not threads

This section shifts the reader from thinking about individual GPU threads to writing Triton programs that operate on blocks of data.

Key terms: block-level programs, thread-level programs, program instance, block size, vectorized, coordination, synchronization, compiler, runtime

2.1.2 The Single Program, Multiple Data model

Explains how Triton applies one program to many data blocks, how that differs from CUDA’s thread-level model, and how a grid assigns blocks to program instances.

Key terms: Single Program, Multiple Data (SPMD), Single Instruction, Multiple Threads (SIMT), warp, blocked SPMD, tiled SPMD, program instance, block, tile, grid, program ID

2.2 A complete Triton kernel: vector addition end to end

This section uses vector addition to show how a Triton kernel’s block-level work and a host-side launch combine to produce a complete GPU computation.

Key terms: SPMD, program instance, block, grid, vector addition, device kernel, host-side wrapper, host, device, grid configuration

2.2.1 Imports and the JIT decorator

This section explains the roles of Triton’s imports and how @triton.jit compiles, specializes, and caches a kernel when it is first launched.

Key terms: triton, triton.language, tl.load, tl.store, @triton.jit, Just-In-Time compilation, first launch, cached, specialization, shapes

2.2.2 Function signature and pointers

This section explains what the vector-addition kernel’s pointer and size parameters mean, and why BLOCK_SIZE is a compile-time constant while vector length remains a runtime argument.

Key terms: x_ptr, y_ptr, z_ptr, pointer, base address, offset, BLOCK_SIZE, tl.constexpr, type annotation, compile-time constant

2.2.3 Getting a unique program ID

Explains how a Triton program instance uses its program ID to identify the block of a vector it handles.

Key terms: program ID, tl.program_id(axis=0), pid, launch grid, BLOCK_SIZE

2.2.4 Calculating the data chunk

Explains how a Triton program finds its vector chunk and how the launch-time block size affects the work assigned to each program.

Key terms: block_start, pid, BLOCK_SIZE, SPMD model, warps

2.2.5 Computing the element offsets

Shows how a Triton program turns positions within its block into indices in the global array.

Key terms: block_start, tl.arange, local indices, absolute indices

2.2.6 Handling edges with a mask

This section explains how a boolean mask lets the final vector-addition block handle offsets beyond the array safely.

Key terms: BLOCK_SIZE, offsets, mask, tl.load, tl.store

2.2.7 Loading data and performing the computation

Explains how each Triton program loads its block of vector data, adds corresponding elements, and stores valid results.

Key terms: tl.load, tl.store, global memory, registers, x_ptr, y_ptr, offsets, BLOCK_SIZE, mask, element-wise addition

2.2.8 The launch grid

Explains how a launch grid assigns one program instance per vector block, adapts to the chosen block size, and lets the same kernel scale to different input sizes.

Key terms: launch grid, n_elements, BLOCK_SIZE, triton.cdiv, lambda, meta, tl.constexpr, program instance, tl.program_id(0), host code

2.2.9 Kernel execution flow

This section traces a vector-addition kernel from the host launch through each program’s work to completion across the GPU.

Key terms: host, launch grid, program instance, program ID, offsets, mask, registers, element-wise addition, output tensor, grid

2.2.10 Benchmarking the kernel

The section shows how to benchmark a Triton vector-addition kernel reliably and interprets its performance against Python and PyTorch.

Key terms: asynchronous execution, torch.cuda.synchronize(), CUDA events, triton.testing.assert_close, reference tensor, atol, rtol, just-in-time compilation, warmup runs, n_elements

2.3 A high-level language with a low-level compiler

This section explains Triton’s high-level programming and low-level compilation duality, while noting that its block-level syntax does not remove every GPU performance concern.

Key terms: high-level syntax, low-level compiler, machine code, block-level syntax, warp divergence, warp

2.3.1 The compilation funnel (advanced)

This subsection traces how Triton progressively compiles a block-level Python kernel into GPU-executable code.

Key terms: @triton.jit, abstract syntax tree, TTIR, TTGIR, LLVM IR, PTX, Triton IR, Triton GPU IR, layout, coalescing

2.3.2 Practical inspection of compilation stages

This section shows how to inspect Triton’s compilation stages locally or in Compiler Explorer to understand and troubleshoot generated GPU code.

Key terms: local cache, compiled kernel object, asm attribute, TTIR, PTX, cubin, TTGIR, block-level logic, thread hierarchy, performance cliffs

2.4 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 1: Introduction to Triton Chapter 3: Fused kernels and parallel patterns →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.