concept hide layer in category deep learning

This is an excerpt from Manning's book Deep Learning for Vision Systems.
This is where the core of the feature learning process takes place. When you look at the hidden layers nodes in the diagram above, you will see that the early layers detect simple patterns in the training to learn low level features (straight lines). Later layers detect patterns within patterns to learn more complex features and shapes. Then patterns within pattern within patterns and so on. This concept will come in handy when we discuss Convolutional Networks (CNNs) in later chapters. For now, know that, in neural networks, we stack hidden layers to learn complex features from each other until we fit our data. So when designing your neural network, if your network is not fitting the data, the answer could be adding more hidden layers.

This is an excerpt from Manning's book Probabilistic Deep Learning: With Python, Keras and TensorFlow Probability.
To get a first idea about an artificial NN, you can better imagine a neuron as a container for a number. The neurons in the input layer are correspondingly holding the numbers of the input data. Such input data could for example be the age (in years), income (in dollars), and height (in inches) of a customer. All neurons in the following layers get the weighted sum of the values from the connected neurons in the previous layer as their input. In general, the different connections aren’t equally important but have weights, which determine the influence of the incoming neuron’s value on the neuron’s value in the next layer. (Here we omit that this input is further transformed within the neuron). DL models are NNs, but they also have a large number of hidden layers (not just three as in the example from figure 1.2).
In this section, you will use the Maximum Likelihood principle to derive the loss function for regression problems. You will start off by revisiting the blood pressure example from chapter 3, where the input was the age of an american healthy woman and the output was a prediction of her systolic blood pressure (sbp). In chapter 3 you have used a simple NN without hidden layers to model a linear relationship between input and output. As the loss function you have used the mean square error. This choice of the loss function was explained by some handwaving arguments, but no hard facts were given to prove that this loss function is a good choice. In this chapter you will see that the MSE as a loss function directly results from the maximum likelihood principle. Further, using the maximum likelihood principle, we can go beyond the mean squared error loss and model data with non constant noise, known under the scary name heteroscedasticity in the statistics community. Don’t be afraid; understanding the maximum likelihood principle makes modeling (not spelling) heteroscedasticity a piece of cake.
Figure 4.12 Simple linear regression as fully connected neural network without hidden layer computing the output directly from the input as out = a · x + b.
![]()
listing 4.6 A fully connected NN with hidden layers, two output nodes and a custom loss to fit the non-linear heteroscatic regression model from equation 4.6
import math def my_loss(y_true,y_pred): #A mu=tf.slice(y_pred,[0,0],[-1,1]) #B sigma=tf.math.exp(tf.slice(y_pred,[0,1],[-1,1])) #C a=1/(tf.sqrt(2.*math.pi)*sigma) b1=tf.square(mu-y_true) b2=2*tf.square(sigma) b=b1/b2 loss = tf.reduce_sum(-tf.math.log(a)+b,axis=0) return loss model = Sequential() #D model.add(Dense(20, activation='relu',batch_input_shape=(None, 1))) model.add(Dense(50, activation='relu')) model.add(Dense(20, activation='relu')) model.add(Dense(2, activation='linear')) model.compile(loss=my_loss,optimizer="adam",metrics=[my_loss])#E