CUDA is NVIDIA’s platform and programming model for general-purpose GPU computing. Its APIs underpin GPU data-analysis libraries including CuPy, cuDF, cuML, and BlazingSQL; the treatment uses NVIDIA APIs while noting that concepts can transfer to AMD and vendor-agnostic tools. section 9.1.2
A CUDA kernel commonly handles one array element per thread. A launch configuration distributes work across blocks and threads; bounds checks protect elements when the total thread count exceeds the array size. GPU code also has to account for data transfers between separate CPU and GPU memory spaces. section 9.2.2 section 9.1.2
Doubling an array
A Numba CUDA kernel uses cuda.grid to find each thread’s position and doubles that array element. Launching it with 50 blocks of 20 threads processes a 1,000-element array; an assertion checks the result. section 9.2.2
Figure 9.1 The CPU and GPU have separate memory spaces. We need to transfer our data to the memory associated with the GPU for computation and then get our results back into the CPU-associated memory, which can cost a substantial amount of time.
The diagram highlights separate CPU and GPU memory spaces and the transfer cost to consider when moving data for computation. section 9.1.2
