chapter ten

10 Combining building blocks to gain more power: Neural networks

 

This chapter covers

  • What is a neural network?
  • What is a perceptron?
  • Using neural networks in a simple application: sentiment analysis.
  • Training neural networks using backpropagation.
  • Potential problems in training neural networks, and techniques that can be used to avoid these problems.
  • How to code the linear regression algorithm in Keras.
  • Applications of neural networks in image recognition, text processing, and more!

In this chapter we learn neural networks. Neural networks are one of the most popular (if not the most popular) machine learning algorithms out there. They are used so much that the field has its own name: deep learning. Deep learning has numerous applications in the most cutting edge areas of machine learning, such as image recognition, natural language processing, medicine, self driving cars, you name it.

Neural networks are meant to, in a broad sense of the word, mimic how the human brain operates. They can be very complicated, as a matter of fact, the following image shows what a neural network looks like.

Figure 10.1. A neural network.

That’s a scary image, right? Lots of nodes, edges, etc. However, there are much simpler ways in which neural networks can be understood. I like to see neural networks as superpositions of linear boundaries which turn into more complicated curves. This chapter is about the intuition, details, and training of neural networks. Let’s begin.

10.1  - The problem - A more complicated alien planet!

10.1.1    Solution - If one line is not enough, use two lines to classify your dataset

10.1.2    Why two lines? Is happiness not linear?

10.1.3    Perceptrons and how to combine them

10.1.4    From discrete perceptrons to continuous perceptrons - a trick to improve our training

10.2  The general scenario - Neural networks

10.2.1    The architecture of a neural network

10.2.2    Bias vs Threshold - Two equivalent ways of describing the constant term in the perceptron

10.3  Training neural networks

10.3.1    Error function - A way to measure how our neural network is performing

10.3.2    Backpropagation - The key step in reducing the error function in order to train the neural network

10.3.3    Potential problems with neural networks - From overfitting to vanishing gradients

10.3.4    Techniques for training your neural network - Dropout, regularization

10.3.5    Different activation functions - Sigmoid, hyperbolic tangent (tanh), and the rectified linear unit (ReLU)

10.3.6    More than one input? No problem, the softmax function is here to help

10.3.7    Hyperparameters - what we fine tune to improve our training

10.3.8    Can neural networks predict values instead of classes? Yes we can! - Neural networks for regression

10.4  How to code a neural network in Keras

10.4.1    Categorizing our data - a way to turn categorical features into numbers

10.4.2    The architecture of a neural network that we’ll use to train this dataset