Softmax converts a model’s C raw class scores into C normalized probabilities. It exponentiates each score and renormalizes the results so they lie between 0 and 1 and sum to 1. The largest score remains the most likely class, while the probability gap expresses confidence. section 2.3.2
Use softmax to connect classification outputs to training and interpretation:
- Pair the resulting probability for the true class with cross-entropy to obtain a classification loss. section 2.3.2
- Treat the values entering softmax as logits and the outputs as probabilities. section 2.3.3
- Apply softmax across the class dimension, such as dim=1 for batched PyTorch outputs. section 2.3.3
def visualize2DSoftmax(X, y, model, title=None):
x_min = np.min(X[:,0])-0.5
x_max = np.max(X[:,0])+0.5
y_min = np.min(X[:,1])-0.5
y_max = np.max(X[:,1])+0.5
xv, yv = np.meshgrid(np.linspace(x_min, x_max, num=20),
np.linspace(y_min, y_max, num=20), indexing=’ij’)
xy_v = np.hstack((xv.reshape(-1,1), yv.reshape(-1,1)))
with torch.no_grad():
logits = model(torch.tensor(xy_v, dtype=torch.float32))
y_hat = F.softmax(logits, dim=1).numpy()
cs = plt.contourf(xv, yv, y_hat[:,0].reshape(20,20),
levels=np.linspace(0,1,num=20), cmap=plt.cm.RdYlBu)
ax = plt.gca()
sns.scatterplot(x=X[:,0], y=X[:,1], hue=y, style=y, ax=ax)
if title is not None:
ax.set_title(title)
visualize2DSoftmax(X, y, model)
Visualize the probability assigned to a class across a two-dimensional input space, alongside the model’s data points. section 2.3.3
