liveBook · Manning

Unified memory

As explained in Vibe Engineering

A unified memory architecture is a hardware design in which processing units such as CPUs and GPUs access a shared memory pool.

Unified memory is a hardware architecture in which CPUs and GPUs share one RAM pool. For developer-local inference, that makes the whole pool available to model weights and avoids the GPU-to-CPU copying overhead common on GPUs with limited VRAM. section 9.3.2

Its practical tradeoffs are visible when choosing local inference hardware:

  • A 32GB Apple Silicon system can run Qwen 2.5 Coder 14B at Q4 while leaving room for development tools. section 9.3.2
  • Systems with 64GB or 128GB can run larger models such as Llama 3.1 70B, but throughput may become unsuitable for interactive completion. section 9.3.2
  • Unified memory handles overflow into the shared pool more gracefully than a 24GB GPU VRAM limit, but it may do so slowly. section 9.3.2
  • The architecture improves capacity flexibility, not raw throughput: comparable NVIDIA hardware can generate tokens roughly twice as fast. section 9.3.2

Choosing a local model

Before downloading a model, enter the machine's unified-memory capacity, model, and quantization level into LLMfit to estimate fit, remaining headroom, and expected throughput. section 9.3.2

Figure 9.3 LLMfit software interface.

Use the LLMfit interface to evaluate whether a model-quantization combination fits available unified memory. section 9.3.2

How Vibe Engineering covers it

How deep it goes
Explains shared-memory benefits, hardware comparisons, model capacity, overflow behavior, and throughput tradeoffs without detailing hardware internals.
Examples
Compares Apple Silicon configurations running Qwen and Llama models with NVIDIA GPUs constrained by VRAM.
What it assumes
Assumes familiarity with CPUs, GPUs, RAM, VRAM, model weights, and quantization.
What you can do after
Evaluate model fit, available headroom, and expected throughput before selecting local inference hardware.
How it connects
Connects unified memory with local inference, GPU, VRAM, model quantization, model weights, and throughput.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.