The sigmoid function maps each real-valued input to a value computed as one divided by one plus the exponential of its negation. section 3.1.2
Sigmoid function
This formula defines the sigmoid used in the SiLU activation. section 3.1.2
In SiLU, each input is passed through the sigmoid and multiplied by itself. Each output depends only on its matching input, so elements can be computed independently in a Triton kernel. section 3.1.2






















