GPU kernel launches return before the GPU necessarily finishes its work. Call torch.cuda.synchronize() before starting and after stopping a CPU wall-clock timer: the first ensures the GPU is idle at the start, and the second waits for queued work to complete. section 2.2.10
import time
import torch
torch.cuda.synchronize()
start = time.time()
z = add(x, y)
torch.cuda.synchronize()
end = time.time()
print(f"elapsed: {end - start:.6f} s")
This timing pattern brackets a Triton kernel call with synchronization so the measured interval includes GPU execution. section 2.2.10
Without synchronization, a wall-clock measurement may capture mostly CPU launch time. CUDA events offer another approach, measuring elapsed time on the GPU timeline. section 2.2.10
