liveBook · Manning

Manual evaluation

As explained in Machine Learning Platform Engineering

Manual inspection is the deliberate examination of a model's outputs and behavior on selected inputs.

Manual evaluation deliberately inspects model behavior on selected inputs alongside automated validation. It exposes unexpected outputs during development and debugging, verifies specific test cases, demonstrates capabilities to stakeholders, and builds confidence before operational use. section 8.1.7

Use manual inspection to:

  • Verify behavior on deliberately selected or previously unseen test images. section 8.1.7
  • Compare predicted objects with the intended targets and investigate unexpected classifications. section 8.1.7
  • Interpret confidence scores, where values near 1 indicate stronger detection confidence and values near 0 indicate weaker confidence. section 8.1.7

Inspecting object-detection outputs

Run inference across images in a samples directory, display each annotated result, and pause for inspection. A generic YOLO model may detect faces as persons and misclassify an ID card as a low-confidence book; repeating the inspection with trained best weights reveals whether training improves the result. section 8.1.7

Listing 8.15 Running inference with the default Nano model
import glob
import cv2
from ultralytics import YOLO

model = YOLO("yolov8n.pt") #1
for file in glob.glob("samples/**.jpg"): #2
    result = model(cv2.imread(file))
    res_plotted = result[0].plot() #3
    cv2.imshow("result", res_plotted) #4
    cv2.waitKey(0)

Provides a runnable inspection loop using Ultralytics YOLO, OpenCV, sample images, plotted detections, and interactive pauses. section 8.1.7

How Machine Learning Platform Engineering covers it

How deep it goes
Demonstrates an image-by-image inference workflow and confidence interpretation, but does not formalize a repeatable evaluation metric.
Examples
Uses sample images, a generic YOLO model, ID-card detections, person detections, and trained best weights.
What it assumes
Assumes image samples and an inference-capable YOLO model, while demonstrating the inspection workflow directly.
What you can do after
Diagnose/debug unexpected predictions and verify improvements by inspecting annotated outputs on selected images.
How it connects
Connects manual inspection with automated validation, model debugging, object detection, inference, and confidence scores.

Also explained in

Compare how 3 books explain manual evaluation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.