chapter eleven

11 CUTLASS

 

This chapter covers

  • Why CUTLASS lets us fuse GEMM workflows beyond cuBLAS
  • Hierarchical tiling that maps grids → threadblocks → warps
  • Extending kernels with CuTe layout and iterator abstractions
  • Tuning tile shapes, pipeline stages, and epilogues for throughput
  • Comparing architectures and precision choices from Ampere to Blackwell
  • Profiling and scaling CUTLASS kernels, including multi-GPU execution

CUTLASS (CUDA Templates for Linear Algebra Subroutines and Solvers) matters when the default GEMM path is almost right but not quite enough. Once a workload needs fused epilogues, custom tile shapes, architecture-specific scheduling, or multi-GPU-aware kernels, the convenience of a black-box library starts to involve a tradeoff with the control needed for performance work. This chapter puts that tradeoff in concrete terms by showing where CUTLASS sits between using cuBLAS as is and writing every kernel detail by hand.

Along the way, we’ll cover how a GEMM is decomposed across the GPU hierarchy, how CUTLASS 3.x assembles kernels from reusable pieces, how precision and architecture choices affect the implementation, and where profiling fits into the workflow when you need to justify using a custom kernel at all.

11.1 Why CUTLASS matters

So far, cuBLAS has been the default answer for GEMM-heavy workloads. We hand it our matrices, and it returns fast, reliable results. For a large class of problems, that is exactly the right choice.

11.1.1 The core idea: Decomposing a GEMM

11.2 Building our first CUTLASS kernel

11.2.1 Defining the kernel’s shape

11.2.2 Composing the kernel with CollectiveBuilder

11.2.3 Launching the kernel from the host

11.3 CuTe: The language of layouts

11.4 Tuning for performance

11.5 Architecture and precision

11.5.1 Across the generations: Ampere vs. Hopper

11.5.2 Narrow precision operations: FP4 on Blackwell

11.6 Profiling and verification

11.7 Beyond a single GPU: Distributed GEMM

Summary