Sigmoid is an activation function that squeezes a model’s single output into the range [0, 1]. This makes the value interpretable as the probability of one class, with 1 − p representing the complementary class. section 3.3.1 section 2.3.2
Use sigmoid for binary classification outputs, such as deciding whether an image is an ankle boot or a t-shirt. For multicategory classification, use softmax instead: its output contains one probability for each category, and the probabilities add up to 1. section 2.3.2 section 2.4.2
A discriminator output
A discriminator can end with one linear unit followed by nn.Sigmoid(). Given a two-value input such as an x,y coordinate, the network returns one value p: interpret p as the probability that the sample is real, and 1 − p as the probability that it is fake. section 3.3.1
Listing 3.3 Creating a discriminator network
import torch.nn as nn
device="cuda" if torch.cuda.is_available() else "cpu" #1
D=nn.Sequential(
nn.Linear(2,256), #2
nn.ReLU(),
nn.Dropout(0.3), #3
nn.Linear(256,128),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(128,64),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(64,1), #4
nn.Sigmoid()).to(device)
The listing shows sigmoid at the end of a PyTorch discriminator, after ReLU hidden layers, dropout layers, and a final one-unit linear layer. section 3.3.1
Listing 2.5 Creating a binary classification model
import torch.nn as nn
device="cuda" if torch.cuda.is_available() else "cpu" #1
binary_model=nn.Sequential( #2
nn.Linear(28*28,256), #3
nn.ReLU(), #4
nn.Linear(256,128),
nn.ReLU(),
nn.Linear(128,32),
nn.ReLU(),
nn.Linear(32,1),
nn.Dropout(p=0.25),
nn.Sigmoid()).to(device) #5
The binary classifier listing shows the same placement: a final linear layer with one output followed by nn.Sigmoid(). section 2.3.2
