A sigmoid, or logistic function, is an S-shaped activation that converts a numeric input into a bounded continuous value. Its smooth transition acts as a soft threshold: outputs near zero and one can represent competing binary outcomes without making an abrupt yes-or-no decision. A single neuron using sigmoid activation is mathematically equivalent to logistic regression; the distinction lies in how the model is trained. section 5.1.4 section 5.2
Changing the curve
The Python implementation accepts a weight, phase, and gain. Changing the weight makes the transition steeper or flatter, while changing the phase shifts where the transition occurs. Plotting several parameter combinations makes these scaling effects visible before using the function inside a model.
Sigmoid connects naturally to classification workflows:
- Use it after a weighted linear combination of features to produce a binary-classification probability. section 5.1.4 section 5.2.5
- Feed it bag-of-words or TF-IDF vectors when classifying text with a single neuron. section 5.1.4 section 5.2.5
- Apply it element-wise for multi-label tagging, where each output independently represents the probability of one label. section 8.2.4
- Prefer binary cross-entropy for binary classification rather than treating the task as regression with mean squared error. section 5.2.5
