liveBook · Manning

cudaDeviceSynchronize

As explained in Grokking GPU Programming

cudaDeviceSynchronize is a CUDA runtime API function that blocks the host until previously issued work on the device has completed.

A CUDA kernel launch returns immediately, so the host can continue while the device is still working. Call cudaDeviceSynchronize() when the host must wait for pending device work to finish—before relying on results or letting the program exit. section 2.5

Use synchronization where completion matters:

  • Wait before the host reads results or exits; otherwise output may be missing or incomplete. section 2.21
  • A synchronization point can also report an error from work that failed during an asynchronous kernel; a bad launch configuration is instead checked with cudaGetLastError() after launch. section 2.5
  • Use synchronization sparingly when tuning performance. section 3.16

Wait before exiting

After launching helloKernel<<<1, 8>>>(), the host could reach return before the eight threads finish printing. Adding cudaDeviceSynchronize() before return makes the host wait. Output may appear complete without it on swforces.com, but real-GPU execution is asynchronous. section 2.1 section 2.21

Figure 2.1 Launch, then wait. The CPU fires the kernel and keeps going while the threads fan out and run in parallel. Threads finish at different times; cudaDeviceSynchronize() makes the CPU wait until the last thread finishes.

The launch-then-wait diagram shows the host waiting for the last thread to finish. section 2.5

How Grokking GPU Programming covers it

How deep it goes
Defines the wait, explains asynchronous behavior and error reporting, and advises sparing use during performance tuning.
Examples
Uses an eight-thread hello kernel and contrasts missing output after early exit with adding a synchronization call.
What it assumes
Assumes kernel launches and host-device roles, while explaining asynchronous execution and synchronization from the beginning.
What you can do after
Operate CUDA programs by waiting after launches before relying on results or exiting.
How it connects
Connects kernel launches with host-device execution, cudaMemcpy, and cudaGetLastError().

Also explained in

Compare how 2 books explain cudaDeviceSynchronize →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.