liveBook · Manning

Memory layout

As explained in Quantization and Fast Inference

Memory layout is the arrangement of data elements at addresses in memory.

Align memory layout with quantization granularity: place groups along the contiguous axis to support coalesced loads. section 3.7

How Quantization and Fast Inference covers it

How deep it goes
Specifies grouping along the contiguous axis for coalesced loads, without detailing implementation.
Examples
Uses group-128 INT4 as a granularity choice and places groups along the contiguous axis.
What you can do after
Operate/configure quantized data layout by grouping along the contiguous axis for coalesced loads.
How it connects
Connects memory layout to quantization granularity and coalesced loads.

Also explained in

Compare how 5 books explain memory layout →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.