liveBook · Manning

Heat map

As explained in Fight Fraud with Machine Learning

A heatmap is a color-coded spatial representation whose intensity at each location encodes a predicted or measured value.

A heatmap overlays color-coded intensity values on an image. In receipt forgery detection, each pixel represents a predicted forgery probability between 0 and 1. section 6.3.5

A computer-vision heatmap supports several practical steps:

  • Resize an input receipt to the model’s training dimensions, run inference, then resize the probability output to the original image dimensions. section 6.3.5
  • Compare predicted regions with ground-truth bounding boxes to inspect localization and expose overfitting on training or test images. section 6.3.5
  • Convert a text-detection heatmap into regions by clustering neighboring pixels into contours and filtering boxes by a threshold. section 6.4.2

Receipt forgery inspection

A segmentation model produces a 256×256 forgery-probability map. Overlaying it on a receipt reveals whether highlighted pixels align with annotated forgery boxes; repeated high probabilities around prices or notes can indicate overfitting. section 6.3.5

mode = "test"
transform = T.Compose([
    T.Resize((256, 256)),  #A
])
image_path = f'./findit2/torchdataset/test/1/<choose a file>.png'  #B
original_image = Image.open(image_path).convert('RGB')
original_size = original_image.size  #C
transformed_image = transform(original_image)  #D
model.eval()  #E
with torch.no_grad():
    input_tensor = T.functional.to_tensor(transformed_image).unsqueeze(0).to(device)  #F
    output = model(input_tensor)['out']
    predictions = torch.argmax(output.squeeze(), dim=0).detach().cpu().numpy() # Convert to numpy array for visualization
predictions_prob = torch.softmax(output, dim=1)[0][1]
predictions_resized = resize(predictions_prob, (original_size[1], original_size[0]),  #G
                             order=0, preserve_range=True, anti_aliasing=False)  #G
 
def draw_bounding_boxes(image, annotations):
    draw = ImageDraw.Draw(image)
    for ann in annotations['regions']:
        shape_attr = ann['shape_attributes']
        x, y, w, h = shape_attr['x'], shape_attr['y'], shape_attr['width'], shape_attr['height']
        draw.rectangle([(x, y), (x+w, y+h)], outline="red", width=4)
    return image
annotations = df[mode][df[mode]["image"]==image_path.split("/")[-1]]["forgery annotations"].iloc[0]
 
original_image = Image.open(image_path).convert('RGB')  #H
drawn_image = draw_bounding_boxes(original_image, annotations)  #I
drawn_image_array = np.array(drawn_image)  #J
fig, ax = plt.subplots(1, 1, figsize=(10, 10))  #K
ax.imshow(drawn_image_array)  #L
heatmap = ax.imshow(predictions_resized, cmap='hot', interpolation='nearest', alpha=0.5)  #M
plt.colorbar(heatmap)
plt.show()

Inspect the inference workflow that generates, rescales, and visualizes a receipt forgery heatmap. section 6.3.5

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains pixel probabilities, resizing, overlays, contour conversion, threshold filtering, and overfitting diagnosis.
Examples
Uses forged receipts, training/test overlays, annotated forgery boxes, and text-detection regions.
What you can do after
Diagnose/debug with it by inspecting localization, comparing predictions with annotations, and identifying spatial overfitting.
How it connects
Connects image segmentation, text detection, bounding boxes, contour detection, multi-task learning, PyTorch, and ONNX.

Also explained in

Compare how 10 books explain heat map →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.