liveBook · Manning

Top-k

Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion.

How 3 Manning books explain it

CUDA for LLMs

Top-k selection identifies the k highest values in a collection and optionally their indices.

How it is used
Uses top-k to route each token to a limited set of mixture-of-experts networks.
How deep it goes
Develops sequential selection, sorted-buffer and heap implementations, then compares their GPU parallelism with reduction and production sorting approaches.
Examples
Works through top-2 selection from eight expert scores and shows PyTorch and CUDA routing paths.
What you can do after
Builds a CUDA top-k scan from scratch and evaluates its synchronization and numerical tradeoffs in MoE routing.

Read how CUDA for LLMs explains top-k →

Build a DeepSeek Model (From Scratch)

Top-k selection retains the k highest-scoring experts for each token and discards the others.

How it is used
Uses sparse routing to reduce per-token computation in Mixture-of-Experts models.
How deep it goes
Explains row-wise selection, negative-infinity masking, and softmax weighting, but stops short of implementing expert-output merging.
Examples
Uses three experts, k=2, the token "The," scores 1, 5, 4, and 60%/40% normalized weights.
What you can do after
Apply it via a router to select experts and compute normalized weights for each token.

Read how Build a DeepSeek Model (From Scratch) explains top-k →

Transformers in Action

Top-k selection is a selection operation that retains the k highest-scoring candidates.

How it is used
Routes tokens to a few experts and narrows retrieved documents to the most relevant candidates.
How deep it goes
Defines selection and weighting with executable gating logic, then extends it to capacity, balancing, and randomized routing.
Examples
Demonstrates PyTorch sparse gating, GShard-style top-2 routing, and RAG retrieval refined with Flashrank or JinaRerank.
What you can do after
Apply it via torch.topk, sparse MoE gates, or retrieval pipelines that rerank and compress selected context.

Read how Transformers in Action explains top-k →

Also covered in

What to know first

Related concepts

Questions about top-k

What is top-k?

Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion.

Which Manning books explain top-k?

CUDA for LLMs, Build a DeepSeek Model (From Scratch) and Transformers in Action.

What should I know before learning top-k?

Manning books that teach top-k usually introduce language model, fine-tuning, Map and information retrieval first.

What is the difference between top-k and top-p?

Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion. Top-p is a language-model generation parameter that restricts token selection to the smallest set of likely tokens whose cumulative probability reaches a specified threshold.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.