WeightedIndex is a Rust random-sampling distribution that chooses an index according to the relative weights supplied to it. In token generation, it turns model probabilities into a stochastic next-token choice rather than always selecting the highest-probability token. section 5.4.2
Sampling the next story token
After softmax converts the final token logits into probabilities, construct a distribution, sample it with a random-number generator, append the selected index to input_ids, and decode the updated sequence. section 5.4.2
let dist = rand::distributions::WeightedIndex::new(&probabilities).unwrap(); let next_token = dist.sample(&mut rng); input_ids.push(next_token as u32); let generated_text = tokenizer.decode(&input_ids, true).unwrap();
The complete sampling expression shows how probabilities, an RNG, token IDs, and tokenizer decoding form one generation step. section 5.4.2
