Mean token accuracy

As explained in Generative AI in Action

Token accuracy is the proportion of tokens predicted correctly in a model output sequence.

Mean token accuracy measures the proportion of tokens predicted correctly in a model output sequence. During fine-tuning, it provides a high-level view of prediction quality alongside training and validation loss. section 9.3.4

Use the metric to:

• Track token-level correctness while training progresses. section 9.3.4
• Read it alongside training loss and validation loss rather than as a standalone performance signal. section 9.3.4

Figure 9.4 Training mean token accuracy

Review the training mean token accuracy visualization when inspecting fine-tuning results. section 9.3.4

How Generative AI in Action covers it

How deep it goes

Defines token correctness and places the metric among training indicators without detailing its calculation procedure.

Examples

Uses a training mean token accuracy figure to visualize fine-tuning results.

What you can do after

Evaluate high-level token prediction quality while fine-tuning a model.

How it connects

Connects mean token accuracy with training loss, validation loss, and fine-tuning.

liveBookmanning.com

Read Generative AI in Action on liveBook Get the book at manning.com

Related concepts

• accuracykind of · Accuracy is the degree to which an output, prediction, or measurement agrees with a correct or reference outcome under a specified criterion.

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.