liveBook · Manning

Face-matching AI model

As explained in Fight Fraud with Machine Learning

Face matching compares facial representations from two images to determine whether they likely depict the same person.

A face-matching AI model compares two facial photographs for identity verification. In online KYC, it powers a duplicate check between two selfies and a selfie–ID face check between a live-style selfie and an identity document photo. This helps detect repeat account sign-ups and confirm that a customer matches the document presented. section 7.1.3 section 7.2

The workflow has three stages: detect and crop the face in each image, turn each crop into a numeric facial embedding, then compare the embeddings with cosine similarity. Directionally closer vectors receive higher similarity scores, while unrelated faces receive lower scores. A score near 1 indicates extremely similar embeddings; a score near 0 indicates unrelated embeddings. section 7.2.1

Figure 7.10 Diagram of a face-matching system comparing faces from two different photos – a selfie and an ID. First, the faces from the two photos are cropped using a face detection model, and then a face recognition model generates embeddings for the faces that are compared numerically for a match.

The pipeline makes the division of labor explicit: detection isolates faces, recognition produces embeddings, and a numerical comparison produces the match signal. section 7.2.1

A recognition model can be trained with a Siamese setup: two copies of a CNN generate embeddings for an image pair, cosine similarity is passed through a sigmoid, and binary cross-entropy compares the prediction with a same-person (1) or different-person (0) label. Training across many positive and negative pairs encourages similar embeddings for the same person and separable embeddings for different people. section 7.2.2

A practical pre-trained implementation uses:

  • MTCNN to find and crop a face from each image. section 7.2.3
  • InceptionResnetV1, pretrained on VGGFace2, to generate a 512-value face embedding. section 7.2.3
  • PyTorch cosine similarity to score the two embeddings. section 7.2.3

Duplicate and document checks

Two Einstein selfies produce a cosine-similarity score of 0.7652. A selfie can also be compared with an ID portrait: an Einstein selfie and ID example scores 61.54%, while a different person’s ID scores 2.52%. An exact copy of the selfie pasted into a fake ID reaches 100%, an abnormally high result that can itself be suspicious. section 7.2.3 section 7.2.4

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains detection, embeddings, cosine similarity, Siamese training, and a pre-trained PyTorch implementation.
Examples
Demonstrates Einstein selfie pairs, Einstein-to-ID comparison, a different-person ID, and a copied face on a fake ID.
What it assumes
Uses pre-trained models while introducing the matching pipeline and high-level recognition-model training.
What you can do after
Apply it via facenet_pytorch and PyTorch to implement duplicate and selfie–ID face checks.
How it connects
Connects face detection, face recognition, embeddings, cosine similarity, Siamese networks, and identity verification.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.