appendix-d

appendix D  CUDA reference: commands, hardware, templates, and bindings

 

D.1 Quick command reference

Check GPU information:

nvidia-smi
nvidia-smi --query-gpu=name,memory.total,compute_cap --format=csv

CUDA compilation:

nvcc file.cu -o output                    # Basic
nvcc file.cu -o output -lcublas           # With cuBLAS
nvcc file.cu -o output -lcublas -lcudnn   # With cuDNN
nvcc file.cu -o output -O3                # Optimized
nvcc file.cu -o output -arch=sm_86        # Specific architecture

Python CUDA extension:

uv run python setup.py build_ext --inplace

MPI execution:

mpirun -np 8 ./program                                    
➥GPUs#1
mpirun -np 8 --mca btl tcp,self ./program                 #2

Profiling:

# Kernel-level profiling (detailed metrics)
ncu --set full -o kernel_profile ./program

# Application-level profiling (timeline view)
nsys profile --trace=cuda,nvtx -o app_profile ./program

# Memory debugging
compute-sanitizer ./program

D.2 Hardware architecture quick reference

The tables below give the memory sizes, latencies, and compute capabilities you will look up most often.

D.2.1 Memory hierarchy

D.2.2 GPU compute capabilities

D.2.3 CUDA core vs. tensor core decision matrix

D.3 CUDA templates and boilerplate guide

D.3.1 CUDA template basics

D.3.2 Common boilerplate patterns

D.3.3 Advanced template patterns

D.3.4 PyTorch C++ extension boilerplate

D.3.5 Template best practices

D.3.6 Complete example: Templated matrix multiply

D.4 Python bindings for CUDA kernels

D.4.1 pybind11: Pure Python bindings