A neural network is a parameterized model that uses layers of neurons to transform inputs into increasingly complex feature combinations. Its nonlinearity grows with network depth, along with the number of weights that must be learned. This makes neural networks a flexible alternative to manually searching through combinations of nonlinear functions. section 7.1.5
For nonlinear regression, the model supports a practical workflow:
- Represent the relationship between inputs and continuous labels through successive neural-network layers. section 7.1.5
- Increase depth when a more complex nonlinear relationship is needed, while recognizing that additional depth introduces more weights to learn. section 7.1.5
- Compare predictions with suitable regression measures such as mean squared error and mean absolute deviation. section 7.1.5
Synthetic nonlinear regression
A small data set can be generated from a damped sinusoidal function with added noise, then divided into training and test sets. Scikit-learn's MLPRegressor fits a network with three hidden layers of 50 units. On the reported test set, the neural network obtains MSE 0.043 and MAD 0.164, compared with 0.027 and 0.131 for a depth-five decision tree. section 7.1.5



