8 Combining building blocks to gain more power: Neural Networks

 

This chapter covers

  • What is a neural network?
  • What is a perceptron?
  • Using neural networks in a simple application: sentiment analysis.
  • Training neural networks using backpropagation.
  • Potential problems in training neural networks, and techniques that can be used to avoid these problems.
  • How to code the linear regression algorithm in Keras.
  • Applications of neural networks in image recognition, text processing, and more!

In this chapter we learn neural networks. Neural networks are one of the most popular (if not the most popular) machine learning algorithms out there. They are used so much that the field has its own name: deep learning. Deep learning has numerous applications in the most cutting edge areas of machine learning, such as image recognition, natural language processing, medicine, self driving cars, you name it.

Neural networks are meant to, in a broad sense of the word, mimic how the human brain operates. They can be very complicated, as a matter of fact, the following image shows what a neural network looks like.

Figure 8.1. A neural network.

That’s a scary image, right? Lots of nodes, edges, etc. However, there are much simpler ways in which neural networks can be understood. I like to see neural networks as superpositions of linear boundaries which turn into more complicated curves. This chapter is about the intuition, details, and training of neural networks. Let’s begin.

8.1   The problem - A more complicated alien planet!

8.1.1   Solution - If one line is not enough, use two lines to classify your dataset

8.1.2   Why two lines? Is happiness not linear?

8.1.3   Perceptrons and how to combine them

8.1.4   From discrete perceptrons to continuous perceptrons - a trick to improve our training

8.2   The general scenario - Neural networks

8.2.1   The architecture of a neural network

8.2.2   Bias vs Threshold - Two equivalent ways of describing the constant term in the perceptron

8.3   Training neural networks

8.3.1   Error function - A way to measure how our neural network is performing

8.3.2   Backpropagation - The key step in reducing the error function in order to train the neural network

8.3.3   Potential problems with neural networks - From overfitting to vanishing gradients

8.3.4   Techniques for training your neural network - Dropout, regularization

8.3.5   Different activation functions - Sigmoid, hyperbolic tangent (tanh), and the rectified linear unit (ReLU)

8.3.6   More than one input? No problem, the softmax function is here to help

8.3.7   Hyperparameters - what we fine tune to improve our training

8.3.8   Can neural networks predict values instead of classes? Yes we can! - Neural networks for regression

8.4   How to code a neural network in Keras

8.4.1   Categorizing our data - a way to turn categorical features into numbers

8.4.2   The architecture of a neural network that we’ll use to train this dataset