GPU memory
GPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
GPU Programming with Triton
Video memory is memory dedicated to storing data for access by a graphics processing unit.
- How it is used
- Uses global memory as the central data-movement constraint in GPU kernel design and optimization.
- How deep it goes
- Explains the hierarchy and DRAM access path, then quantifies memory transfers in a fused computation.
- Examples
- Uses masked vector addition and fused SiLU multiplication to illustrate memory access and traffic reduction.
- What you can do after
- Apply it via Triton to reason about masked loads and stores and reduce intermediate global-memory transfers through fusion.
Local Image Generation
VRAM is the dedicated memory available to a GPU for storing data needed during computation.
- How it is used
- Constrains batch settings, image dimensions, model selection, and diffusion-model fine-tuning.
- How deep it goes
- Explains practical memory trade-offs and training estimates without detailing GPU memory internals.
- Examples
- Uses coffee-image batches, landscape dimensions, a 2048×2048 tortoise image, and Flux memory estimates.
- What you can do after
- Operate image-generation settings by balancing batch size and dimensions against available VRAM.
CUDA for LLMs
Device memory is GPU-resident storage accessible to device-side computation such as CUDA kernels.
- How it is used
- Uses device memory to hold data for GPU kernels while host code manages allocation and transfers.
- How deep it goes
- Explains allocation, transfers, cleanup, and memory-hierarchy tradeoffs, with a concrete account of repeated global-memory reads.
- Examples
- Illustrates device allocation with vector addition and neural-network parameters, then examines repeated reads in softmax.
- What you can do after
- Apply CUDA runtime memory allocation, transfer, and cleanup to a basic kernel workflow.
Classic Game Programming on the NES: Make your own retro video game
Video memory (VRAM) is memory accessed by graphics hardware to store data needed to generate a display.
- How it is used
- Supports NES screen drawing by holding tile, sprite, and name-table data for the PPU.
- How deep it goes
- Explains VRAM ownership, PPU-port access, vBlank timing, and frame-limited update strategies without detailing port code.
- Examples
- Uses title-screen pattern loading, 32-by-30 tile name tables, and background tile or color animation.
- What you can do after
- Operate/configure it by loading patterns before display and scheduling small VRAM updates during vBlank.
Read how Classic Game Programming on the NES: Make your own retro video game explains GPU memory →
Grokking GPU Programming
Device memory is memory located on a GPU and accessible to GPU computations.
- How it is used
- Supplies storage for GPU kernel inputs and outputs, with transfers connecting it to CPU host memory.
- How deep it goes
- Defines device and global memory, outlines allocation and transfers, and contrasts global with shared memory without detailing hardware internals.
- Examples
- Uses an integer array passed to a kernel and matrix multiplication that reuses global-memory data through tiling.
- What you can do after
- Operate on GPU data by allocating device memory, transferring inputs and results, and passing device pointers to kernels.
Vibe Engineering
VRAM is memory on a graphics-processing unit used to store graphics and compute workloads such as model parameters.
- How it is used
- Frames VRAM as the capacity constraint for developer-local inference on discrete GPUs.
- How deep it goes
- Explains capacity limits, memory placement, quantization, and sizing tools, but stops short of hardware-level GPU internals.
- Examples
- Uses RTX 4060, RTX 4070, RTX 4090, 7B Q4 models, and LLMfit as concrete sizing examples.
- What you can do after
- Operate/configure local inference by matching model quantization and size to VRAM, headroom, and expected throughput.
Also covered in
- The Creative Programmer
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- Machine Learning Platform Engineering
- LLMs in Production
- Transformers in Action
- Embedded Software with RustIn early access (MEAP)
- Self-Hosting LLMs at ScaleIn early access (MEAP)
- Designing Deep Learning Systems: A guide for software engineers
What to know first
- threadA thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- matrix-matrix multiplicationMatrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
- quantizationModel quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
Related concepts
- memory hierarchypart of · A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
- cudaMallocused by · cudaMalloc is a CUDA runtime API function that allocates memory in device memory.
- shared memorycontrasts with · Shared memory is a memory region accessible to multiple processes, threads, or other concurrent execution contexts for sharing data.
- gradient accumulationaddressed by · Gradient accumulation is a training technique that combines gradients from multiple smaller batches before applying an optimizer update to model parameters.
Questions about GPU memory
What is GPU memory?
GPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
Which Manning books explain GPU memory?
GPU Programming with Triton, Local Image Generation, CUDA for LLMs, Classic Game Programming on the NES: Make your own retro video game, Grokking GPU Programming and Vibe Engineering.
What should I know before learning GPU memory?
Manning books that teach GPU memory usually introduce thread, matrix-matrix multiplication, quantization and attention first.
What is the difference between GPU memory and RAM?
GPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation. RAM is volatile, directly addressable computer memory that temporarily stores data and instructions for use by running programs.
What is GPU memory part of?
GPU memory is part of memory hierarchy. A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
What is GPU memory also called?
GPU memory is also called VRAM, global memory, video memory, device memory or Video RAM.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





