A probability distribution turns a set of model scores into likelihoods for possible outcomes. Here, softmax exponentiates each score and divides by their total, producing values that add up to 1. The resulting values make inference output interpretable and usable for ranking or randomized selection. section 5.2.2 section 5.4.2
Use a probability distribution after model inference to choose or report outcomes:
- For image classification, associate each normalized probability with a class label, sort the pairs, and report the top predictions as the model’s most likely matches. section 5.2.2
- For text generation, use the probabilities over next tokens as weights in a WeightedIndex distribution, then sample one token to continue the sequence with controlled randomness. section 5.4.2
Classifying an image
A model’s raw output is converted from bytes to floating-point values, normalized with softmax, then ranked alongside class labels. A Persian cat image yields a leading Persian-cat probability, followed by lower-probability alternatives. section 5.2.2
fn softmax(logits: &[f32]) -> Vec<f32> {
let logits_array = Array::from_vec(logits.to_vec());
let exp_logits = logits_array.mapv(f32::exp);
let sum_exp = exp_logits.sum();
(exp_logits / sum_exp).to_vec()
}
The reusable softmax function computes exponentials, sums them, and divides each value by that sum to form next-token probabilities. section 5.4.2
Generating the next story token
After selecting the logits for the latest token position, softmax produces a distribution across the vocabulary. Weighted sampling selects a likely next token without always choosing the single highest-scoring one. section 5.4.2
