liveBook · Manning

Asynchronous programming

As explained in Grokking GPU Programming

Asynchronous execution allows an operation to proceed without requiring the caller to wait for its completion before continuing.

A CUDA kernel launch is asynchronous: the CPU starts GPU work without waiting for it to finish. The GPU can process data while the CPU continues; cudaDeviceSynchronize() makes the CPU wait until the GPU work is done. section 2.17

Squaring array elements

A kernel launches one thread per element to square values in device memory. Calling cudaDeviceSynchronize() after the launch gives the CPU a clear wait point, so kernel errors surface there during debugging. A blocking device-to-host copy also waits for earlier GPU work, so an extra synchronization before that copy is not required. section 2.17

Figure 2.9 Step 3 — Compute: the GPU takes over. The CPU launches the kernel and steps back to wait; all four threads fire at once, each one reading its value from device memory, squaring it, and writing the result back (the up and down arrows), all in parallel. When the last thread finishes, cudaDeviceSynchronize() returns and the CPU carries on.

Trace the CPU launching the kernel, GPU threads processing values, and the CPU resuming after synchronization. section 2.17

How Grokking GPU Programming covers it

How deep it goes
Traces kernel launch, explicit synchronization, and blocking-copy behavior through a concrete execution sequence.
Examples
Uses a kernel that squares array elements with one GPU thread per element.
What you can do after
Apply it via CUDA by adding an explicit synchronization point to surface kernel errors during debugging.
How it connects
Connects CUDA kernels with device memory, synchronization, and blocking memory copies.

Also explained in

Compare how 37 books explain asynchronous programming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.