CUDA for LLMs cover
preface

preface

 

I wrote this book because it is the one I wanted when I was starting out. I did not come to CUDA as someone with 10 years of performance engineering and distributed systems experience. I came to it as a curious person who wanted to understand what was happening underneath the deep learning tools I was already using.

When I first tried to learn CUDA for deep learning, I often found myself missing the connective tissue between ideas. I could find explanations of CUDA mechanics, and I could find examples of sophisticated kernels, but I still had to work out how the pieces lined up: how threads map to tensors, how memory access patterns become performance problems, how reductions show up inside ordinary model operations, and why attention changes the shape of the whole optimization problem.

That process of connecting the dots became this book. Many of these concepts were not perfectly clear to me the first time I encountered them. Writing the chapters forced me to slow down, build the examples, test the kernels, check the performance claims, and teach each idea back carefully enough that I knew I understood it. The result is a path through CUDA that follows the order in which the pieces became clear to me.