liveBook · Manning

Overfitting

As explained in Fight Fraud with Machine Learning

Overfitting is a modeling condition in which a model captures training-specific patterns that reduce performance on unseen data.

Overfitting occurs when a model learns training-specific patterns that fail to transfer to unseen data. In receipt forgery detection, training heatmaps can align with forgery boxes while test heatmaps miss the regions and repeatedly assign high probabilities to superficial areas such as item prices or receipt notes. section 6.3.5

Recognize overfitting by comparing model behavior across splits:

  • Compare training and test predictions against the actual forgery bounding boxes; weaker overlap on test receipts indicates poorer generalization. section 6.3.5
  • Inspect where predicted probabilities concentrate; recurring high-probability regions unrelated to the forgery suggest reliance on training-set artifacts. section 6.3.5
  • Reduce overfitting by adding labeled data when available or by revisiting the training pipeline and model architecture; multi-task learning adds text detection as a related auxiliary task. section 6.3.5 section 6.4.5

Receipt forgery heatmaps

A segmentation model closely matches forgery boxes on training receipts but highlights item prices and textual notes on test receipts. A multi-task model trained to detect both forgery regions and receipt text produces stronger overlaps on both splits. section 6.3.5 section 6.4.5

mode = "test"
transform = T.Compose([
    T.Resize((256, 256)),  #A
])
image_path = f'./findit2/torchdataset/test/1/<choose a file>.png'  #B
original_image = Image.open(image_path).convert('RGB')
original_size = original_image.size  #C
transformed_image = transform(original_image)  #D
model.eval()  #E
with torch.no_grad():
    input_tensor = T.functional.to_tensor(transformed_image).unsqueeze(0).to(device)  #F
    output = model(input_tensor)['out']
    predictions = torch.argmax(output.squeeze(), dim=0).detach().cpu().numpy() # Convert to numpy array for visualization
predictions_prob = torch.softmax(output, dim=1)[0][1]
predictions_resized = resize(predictions_prob, (original_size[1], original_size[0]),  #G
                             order=0, preserve_range=True, anti_aliasing=False)  #G
 
def draw_bounding_boxes(image, annotations):
    draw = ImageDraw.Draw(image)
    for ann in annotations['regions']:
        shape_attr = ann['shape_attributes']
        x, y, w, h = shape_attr['x'], shape_attr['y'], shape_attr['width'], shape_attr['height']
        draw.rectangle([(x, y), (x+w, y+h)], outline="red", width=4)
    return image
annotations = df[mode][df[mode]["image"]==image_path.split("/")[-1]]["forgery annotations"].iloc[0]
 
original_image = Image.open(image_path).convert('RGB')  #H
drawn_image = draw_bounding_boxes(original_image, annotations)  #I
drawn_image_array = np.array(drawn_image)  #J
fig, ax = plt.subplots(1, 1, figsize=(10, 10))  #K
ax.imshow(drawn_image_array)  #L
heatmap = ax.imshow(predictions_resized, cmap='hot', interpolation='nearest', alpha=0.5)  #M
plt.colorbar(heatmap)
plt.show()

Run the test-image inference and heatmap workflow to inspect predictions on an unseen receipt and compare highlighted regions with annotations. section 6.3.5

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains overfitting through train/test heatmaps and demonstrates multi-task mitigation, without developing a general theoretical treatment.
Examples
Uses receipt forgery bounding boxes, probability heatmaps, item-price regions, textual notes, and auxiliary text-detection predictions.
What it assumes
Assumes familiarity with image segmentation, training and test splits, model predictions, and probability heatmaps.
What you can do after
Diagnose overfitting by comparing train/test heatmaps and apply multi-task learning or pipeline changes to improve generalization.
How it connects
Connects overfitting to training and test sets, generalization, segmentation, CNNs, and multi-task learning.

Also explained in

Compare how 33 books explain overfitting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.