A logit is a raw, unnormalized score that a neural network assigns to each possible next token. It becomes a probability only after applying softmax. section 5.4.2
Selecting the next token
After inference, read the model's "logits" output, convert its bytes to f32 values, isolate the final token's vocabulary slice, and apply softmax. The resulting probabilities can drive weighted sampling of the next token. section 5.4.2
let output_data = exec_context.get_output("logits").unwrap().data();
let output_f32 = bytes_to_f32_vec(&output_data);
Inspect the inference output and convert the raw logits bytes into f32 values before selecting the relevant token scores. section 5.4.2
