Perplexity measures how surprised a language model is by a sequence of text. Lower values generally indicate more natural language because the model assigns higher probability to the sequence. section 1.1.2
Use it as a fast, inexpensive intrinsic metric for rapid iteration and regression testing. It evaluates how text is written, not whether the output completes its task, remains faithful to data, or satisfies application-specific requirements. section 1.1.2
Interpret perplexity alongside other evaluation signals:
- Compare it with coherence and consistency checks when evaluating generated text. section 1.1.2
- Add human evaluation or LLM-as-a-Judge when correctness, relevance, or task success matters. section 1.1.2
Figure 1.5 Reference-free evaluation workflow. Quality signals are defined for the task. The AI generates outputs for test inputs. Metrics assess each output’s intrinsic properties without reference answers. Scores reveal quality patterns and failure modes.
Places perplexity within reference-free evaluation, where intrinsic metrics assess generated outputs without reference answers and expose quality patterns or failure modes. section 1.1.2








