CUDA is NVIDIA’s parallel computing platform and programming model for general-purpose GPU work. It lets a host program launch computations on a GPU, including workloads in deep learning and scientific computing. CUDA C++ provides a low-level way to write these GPU kernels. section 1.2 section 1.1.1
A CUDA kernel runs work in parallel on the device after being launched by the host. Threads are the basic units of execution: each can process its own piece of data. Threads form warps; on NVIDIA GPUs, a warp contains 32 threads. Blocks group work within a grid, and the GPU schedules blocks across streaming multiprocessors. section 1.3.3 section 1.5
Figure 1.6 The relationship between the software execution model (threads, blocks, and grids) and the underlying GPU hardware (cores, streaming multiprocessors, and device). Each level of the software model corresponds to a specific layer of GPU hardware responsible for executing it..
The execution hierarchy connects threads, blocks, and grids to the GPU hardware that runs them. section 1.3.3
Low-level CUDA programming gives developers control over how work is divided into threads and blocks, how data moves between global and shared memory, and how threads synchronize. That control can deliver excellent performance, but indexing, memory alignment, and parallel coordination add complexity. section 1.2.1
Matrix multiplication
A framework call such as torch.matmul(A, B) can provide efficient GPU execution without exposing its implementation. A CUDA implementation offers more control over execution, but even this operation can require hundreds of lines to manage its low-level details. section 1.2.1
Triton sits between high-level frameworks and CUDA: it lets developers write GPU kernels in Python while its compiler manages much of the thread-level work. A Triton program instance corresponds to what CUDA calls a block. On NVIDIA hardware, Triton requires CUDA 11.8 or later and compiles kernels to PTX; NVIDIA’s ptxas then lowers PTX to GPU-specific machine code. section 1.3.3 section 1.1.1 section 1.2.1






