liveBook · Manning

Qualitative assessment

Qualitative assessment evaluates a system or its outputs through descriptive judgments of their characteristics rather than primarily through numerical measurements.

How 2 Manning books explain it

The Art of AI Product Development

Qualitative evaluation assesses a system through descriptive observations, judgments, and feedback rather than primarily numerical measurements.

How it is used
Uses early user feedback to establish a search-system baseline before teams optimize retrieval or other system components.
How deep it goes
Progresses from manual vibe checks to user studies and structured judgments, while deferring advanced metrics until data and optimization mature.
Examples
Uses queries about advanced-plan feedback and API integration case studies, plus live retrieval of customer feedback for case studies.
What you can do after
Evaluate search usefulness, identify broad improvement areas, and establish a baseline for future changes.

Read how The Art of AI Product Development explains qualitative assessment →

LLM Customization and Fine-Tuning

Assessment of model outputs by examining their characteristics rather than relying only on numerical scores.

How it is used
Evaluates model outputs alongside quantitative metrics when ground truth alone cannot capture quality, safety, or fairness.
How deep it goes
Examines responses for helpfulness, tone, risk sensitivity, and specificity without prescribing a formal scoring rubric.
Examples
Contrasts base, teacher, and distilled-student answers to a Mac company-VPN prompt.
What you can do after
Evaluate responses for structure, specificity, and actionability to judge whether a student model suits the task.

Read how LLM Customization and Fine-Tuning explains qualitative assessment →

Also covered in

What to know first

Related concepts

Questions about qualitative assessment

What is qualitative assessment?

Qualitative assessment evaluates a system or its outputs through descriptive judgments of their characteristics rather than primarily through numerical measurements.

Which Manning books explain qualitative assessment?

The Art of AI Product Development and LLM Customization and Fine-Tuning.

What should I know before learning qualitative assessment?

Manning books that teach qualitative assessment usually introduce accuracy first.

What is the difference between qualitative assessment and quantitative evaluation?

Qualitative assessment evaluates a system or its outputs through descriptive judgments of their characteristics rather than primarily through numerical measurements. Quantitative evaluation is the measurement and analysis of numerical system behavior or outcomes against defined inputs, criteria, or desired results.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.