liveBook · Manning

Sigmoid

As explained in Fight Fraud with Machine Learning

A sigmoid function is a smooth S-shaped function that maps real-valued inputs to a bounded interval, commonly zero to one.

A sigmoid is a smooth S-shaped activation function that squashes real-valued inputs into a bounded range, commonly between zero and one. This makes it useful when a neural network must produce a binary-classification probability, although its gradients become close to zero at extreme inputs. section 4.1.1

Use sigmoid to:

  • Turn a weighted sum into a probability in a single-layer neural network equivalent to logistic regression. section 4.1.2
  • Produce a same-person prediction from cosine similarity between face embeddings in a Siamese network. section 7.2.2
  • Choose between activation functions with practical trade-offs: sigmoid supports bounded outputs, while ReLU avoids its extreme-input gradient problem and often converges faster. section 4.1.1

Face matching

Two copies of a CNN generate embeddings for a pair of face images. Cosine similarity passes through a sigmoid to yield a value from zero to one, which is compared with a label using binary cross-entropy. section 7.2.2

Figure 4.5 Some examples of activation functions commonly used in deep neural networks – Sigmoid, TanH, and ReLU.

Compare sigmoid with TanH and ReLU as commonly used deep-learning activation functions. section 4.1.1

Figure 7.12 Demonstration of how a face recognition model is trained using the Siamese network. Two copies of a CNN model produce two different embeddings for two different faces (images). These embeddings produce a cosine similarity score. This score is passed through a sigmoid function and compared to the label (1 if faces are from same person and vice versa) using a loss function such as binary cross entropy.

Trace sigmoid’s position in a Siamese face-recognition training pipeline. section 7.2.2

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains bounded outputs, classification use, gradient limitations, and Siamese placement, but does not provide the sigmoid formula or implementation code.
Examples
Uses dog-versus-cat classification, logistic regression, and paired-face similarity as concrete sigmoid applications.
What you can do after
Evaluate sigmoid’s suitability for bounded classification outputs and recognize its gradient trade-off against ReLU.
How it connects
Links sigmoid to activation functions, neural networks, logistic regression, binary cross-entropy, gradient descent, Siamese networks, CNNs, and cosine similarity.

Also explained in

Compare how 23 books explain sigmoid →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.