liveBook · Manning

CUDA data lifecycle

As explained in Grokking GPU Programming

The CUDA data lifecycle is the five-stage GPU workflow of allocating device memory, copying data in, computing, copying results out, and freeing device memory.

A CUDA program moves data between CPU (host) memory and GPU (device) memory. Its common five-step workflow is allocate device memory, copy input to the GPU, compute, copy results back, and free device memory. section 2.14

Figure 2.6 The CUDA data lifecycle, five steps from CPU to GPU and back: allocate GPU memory, copy data in, run the kernel, copy results out, and free the memory.

Follow the data’s trip from CPU to GPU and back through the five steps. section 2.14

For an array, the steps use these CUDA operations:

  • Allocate device memory with cudaMalloc. section 2.14
  • Copy input from host to device with cudaMemcpy and cudaMemcpyHostToDevice. section 2.14
  • Launch a kernel to compute, then call cudaDeviceSynchronize() before relying on its results. section 2.19
  • Copy results back with cudaMemcpy and cudaMemcpyDeviceToHost. section 2.19
  • Release device memory with cudaFree when it is no longer needed. section 2.19

Squaring an array

For a small array of numbers, allocate a device array, copy the input over, launch a kernel that squares each element, wait for the kernel, copy the squares back, and free the device array. A missing copy-in can leave the kernel operating on uninitialized GPU memory, producing zeros or garbage instead of correct results. section 2.14 section 2.19

How Grokking GPU Programming covers it

How deep it goes
Explains all five runtime steps and synchronization, using a single-block example limited to at most 1,024 threads.
Examples
Uses an array-squaring program and a missing-copy-in bug that produces incorrect output.
What you can do after
Apply it via CUDA runtime calls to allocate device memory, transfer data, compute, retrieve results, and release memory.
How it connects
Connects host and device memory, memory transfers, CUDA kernels, and synchronization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.