CPU cache

As explained in 100 Go Mistakes and How to Avoid Them

A CPU cache is small, fast hardware memory that stores recently accessed data and instructions near a processor core.

A common mistake is to assume that reading the same elements must cost the same regardless of how the surrounding data is laid out. A CPU cache keeps recently accessed data and instructions closer to a core than main memory. On one example processor, L1 access takes about 1 ns, while main-memory access takes roughly 50–100 times longer. Those figures are hardware-specific, but the performance gap makes data layout worth examining. section 12.1.1

The CPU usually fetches a memory block into a cache line rather than fetching one value at a time. Reading nearby values makes more use of each line; this is spatial locality. Cache placement also matters: a set-associative cache partitions its lines into sets, so repeatedly accessing addresses that compete for a set can evict useful data. section 12.1.5 section 12.11

Same columns, different row width

Consider two Go matrices, `[][512]int64` and `[][513]int64`. For each row, sum only columns 0 through 7: ```go for i := range matrix { for j := 0; j < 8; j++ { sum += matrix[i][j] } } ``` The loop reads the same number of values either way, yet a benchmark that repeatedly reused its matrix found the 513-column version about 50% faster on one machine. When each benchmark used a new matrix, the difference disappeared. The row width changes memory addresses and therefore how blocks compete for cache placement; the result is not a rule that 513 columns are always faster. section 12.1.5

Figure 12.10 Memory address 1000000000000 replaces an existing cache line in set 0.

The illustrated replacement shows why a cache with available capacity can still lose a useful line when addresses map to the same set. section 12.1.5

When investigating a CPU-bound Go loop:

• Favor predictable access patterns and nearby data when the computation permits; a unit stride can use cache lines more effectively than a non-unit stride. section 12.11
• Consider cache-set conflicts if a repeated access pattern performs unexpectedly; benchmark the actual data layout rather than assuming fewer allocated elements means faster access. section 12.1.5
• In concurrent code, account for caches not being shared uniformly across cores and for performance losses from false sharing. section 12.11

How 100 Go Mistakes and How to Avoid Them covers it

How deep it goes

Explains cache levels, cache-line transfers, and set-associative placement through an address-layout example.

Examples

Compares Go matrices with 512- and 513-element rows while summing only their first eight columns.

What it assumes

Assumes familiarity with Go loops and benchmarks while introducing the cache hierarchy and placement policy.

What you can do after

Diagnose/debug with it when data layout or repeated access produces unexpected benchmark results.

How it connects

Connects CPU caches to cache lines, spatial locality, stride, set-associative caches, and false sharing.

liveBookmanning.com

Read 100 Go Mistakes and How to Avoid Them on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Fast Python: High performance techniques for large datasets
• 

  liveBookmanning.com
  LatencyA CPU cache is a small, fast memory that stores copies of data likely to be accessed by a CPU.

Compare how 3 books explain CPU cache →

Related concepts

• L1 cacheincludes · An L1 cache is a small, low-latency first-level cache located closest to a processor core or execution unit.
• memory hierarchypart of · A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
• cache linehas part · A cache line is a fixed-size block of contiguous memory that a processor cache transfers and manages as a unit.
• memorycontrasts with · Memory is a storage resource or mechanism that retains data or computational state for access during computation or later use.
• data-oriented programmingused by · Data-oriented programming is an approach to structuring software around data representations, access patterns, and operations suited to the data's characteristics and intended use.

This explainer was generated by Manning's knowledge graph from the text of 100 Go Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.