The hyperbolic tangent, or tanh, is a squashing activation function that maps real-valued neuron inputs to values between -1 and 1. It transforms the weighted input and bias before the next layer processes it. section 9.7
Using tanh in a hidden layer
A neural-network forward pass computes z1 = X.dot(W1) + b1, then applies a1 = np.tanh(z1). The resulting hidden-layer activations feed a later logits calculation for classification. section 9.7
def logits_function(p):
W1 = p[0: n_inputs * n_hidden].reshape((n_inputs, n_hidden))
b1 = p[n_inputs * n_hidden: (n_inputs + 1) * n_hidden].reshape((n_hidden,)) #1
W2 = p[(n_inputs +1) * n_hidden: -n_classes].reshape((n_hidden, n_classes)) #2
b2 = p[-n_classes:].reshape((n_classes,)) #3
z1 = X.dot(W1) + b1 #4
a1 = np.tanh(z1) #4
logits = a1.dot(W2) + b2 #5
return logits #5
Inspect the complete logits function, including weight reshaping, bias addition, tanh activation, and output-logit computation. section 9.7
