CUDA kernel
A CUDA kernel is a function launched on a GPU for execution by multiple threads in parallel.
CUDA for LLMs
A kernel is a function launched for parallel execution by many GPU threads.
- How it is used
- Introduces kernels as the GPU execution mechanism for parallelizing deep-learning operations and extending framework workloads.
- How deep it goes
- Develops launch syntax, indexing, bounds checks, memory management, and naive implementations before motivating fusion and optimization.
- Examples
- Works through vector addition, two-dimensional matrix addition, neural-network passes, and naive attention.
- What you can do after
- Builds kernels from scratch, launches them with appropriate indexing and bounds checks, and verifies their results.
Grokking GPU Programming
A CUDA kernel is a function launched on a GPU to execute concurrently across many GPU threads.
- How it is used
- Uses kernels to express GPU work as the same instructions executed concurrently by many threads.
- How deep it goes
- Explains launches, indexing, synchronization, and bounds checks, then walks through tiled kernel layout and its correctness requirements.
- Examples
- Uses an eight-thread greeting and a tiled matrix multiplication kernel with one thread per tile element.
- What you can do after
- Build a CUDA kernel from scratch and launch it with threads assigned to work.
What to know first
- C++C++ is a statically typed, general-purpose programming language that supports procedural, object-oriented, generic, and low-level systems programming.
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
- CUDACUDA is NVIDIA’s parallel computing platform and programming model for performing general-purpose computations on NVIDIA GPUs.
- inner productAn inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.
Related concepts
- global function qualifieruses · In CUDA, __global__ is a function qualifier for kernels that can be launched from host code and execute on a device.
- threaduses · A thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- bounds checkinguses · Bounds checking verifies that an index, access, or operation falls within the permitted limits of a data structure, memory region, or range.
- CUDA kernel launch syntaxconfigured by · CUDA kernel launch syntax uses triple angle brackets to specify a kernel's execution configuration, including its grid and blocks of threads.
Questions about CUDA kernel
What is CUDA kernel?
A CUDA kernel is a function launched on a GPU for execution by multiple threads in parallel.
Which Manning books explain CUDA kernel?
CUDA for LLMs and Grokking GPU Programming.
What should I know before learning CUDA kernel?
Manning books that teach CUDA kernel usually introduce C++, CPU, CUDA and inner product first.
What kinds of CUDA kernel are there?
Kinds of CUDA kernel include generated kernel.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

