liveBook · Manning

Throughput-optimized design

As explained in GPU Programming with Triton

A throughput-optimized design maximizes the amount of work completed over a given period, often by executing many tasks concurrently.

A throughput-optimized design aims to complete many tasks over a period of time rather than minimize the time for one task. GPUs devote most of their chip area to simple arithmetic units and run many threads simultaneously; an individual GPU thread may be slower than a CPU thread, while aggregate throughput can be much greater. Independent tensor additions, multiplications, and normalizations are examples of work that can benefit from this approach. section 1.3.2

Figure 1.5 GPUs dedicate more of their area to compute and less to control and caching.
Growth in training data

Compare how CPU and GPU chip area is allocated to compute, caching, and control. section 1.3.2

How GPU Programming with Triton covers it

How deep it goes
Compares CPU and GPU resource allocation and explains how many slower GPU threads can deliver greater aggregate throughput.
Examples
Uses tensor addition, multiplication, and normalization as independently processable workloads.
What you can do after
Evaluate why GPUs suit workloads with many independent tasks rather than judging performance by one thread.
How it connects
Contrasts with latency-optimized design and connects to concurrency, parallelism, and data-parallel tensor operations.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.