liveBook · Manning

Asynchronous programming

As explained in GPU Programming with Triton

Asynchronous execution allows a processor to issue work and continue without waiting for that work to finish.

A Python call can enqueue a Triton kernel on the GPU and return before the GPU finishes. The CPU and GPU advance independently, so wall-clock timing around the call may measure launch time rather than completed work. Synchronize before reading the end time to ensure the GPU queue has finished. section 2.2.10

import time
import torch

torch.cuda.synchronize()
start = time.time()
z = add(x, y)
torch.cuda.synchronize()
end = time.time()
print(f"elapsed: {end - start:.6f} s")

Compare wall-clock readings taken around a kernel launch with synchronization before and after the launch. section 2.2.10

How GPU Programming with Triton covers it

How deep it goes
Defines queued GPU execution and independent CPU/GPU clocks, then demonstrates synchronization for completed-work timing.
Examples
Compares naive wall-clock timing with timing that brackets a Triton kernel launch using torch.cuda.synchronize.
What it assumes
Assumes familiarity with launching a Triton kernel and measuring Python wall-clock time.
What you can do after
Diagnose misleading kernel timings by synchronizing around launches or using triton.testing.do_bench.
How it connects
Connects GPU kernel launches with synchronization and kernel benchmarking.

Also explained in

Compare how 37 books explain asynchronous programming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.