GPU device memory is storage on the GPU that GPU computations can access. A kernel’s pointer must refer to device memory, not regular CPU (host) memory. CUDA allocates device memory with cudaMalloc and transfers data with cudaMemcpy. section 2.10 section 3.6
Global memory is the large, widely accessible device-memory tier: threads across the grid can reach it, but it is slower than shared memory. Tiling speeds matrix multiplication by loading data from global memory into shared memory, where threads in a block can reuse it. section 3.6 section 12.2
A typical CUDA program handles device memory in this order:
- Allocate GPU memory, then copy input data from host memory to the device. section 2.24
- Launch the kernel with device-memory pointers; synchronize before relying on its results. section 2.10 section 2.24
- Copy results back to the host and free the GPU memory. section 2.24
A pointer from the wrong side
Passing a regular host-memory pointer to a kernel that expects device memory is a mistake. Allocate the array on the GPU and transfer its contents before using it in the kernel. section 2.10 section 2.24





