liveBook · Manning

Sigmoid

As explained in GPU Programming with Triton

The sigmoid function commonly refers to the logistic function, which maps a real-valued input to 1 divided by 1 plus the exponential of its negation.

The sigmoid function maps each real-valued input to a value computed as one divided by one plus the exponential of its negation. section 3.1.2

Sigmoid function
\[\sigma(x) = \frac{1}{1 + e^{-x}}.\]

This formula defines the sigmoid used in the SiLU activation. section 3.1.2

In SiLU, each input is passed through the sigmoid and multiplied by itself. Each output depends only on its matching input, so elements can be computed independently in a Triton kernel. section 3.1.2

How GPU Programming with Triton covers it

How deep it goes
Defines sigmoid mathematically and applies it in SiLU, without deriving its properties or implementation internals.
Examples
Demonstrates sigmoid through SiLU outputs computed independently for elements of an input vector.
What you can do after
Apply it via Triton as part of an element-wise SiLU kernel.
How it connects
Connects sigmoid to SiLU, element-wise operations, and Triton kernels.

Also explained in

Compare how 23 books explain sigmoid →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.