chapter nine

9 Artificial neural networks

 

This chapter covers

  • Understanding the inspiration and intuition of artificial neural networks
  • Identifying problems that can be solved with artificial neural networks
  • Understanding and implementing forward propagation using a trained network
  • Understanding and implementing backpropagation to train a network
  • Designing artificial neural network architectures to tackle different problems

9.1 What are artificial neural networks?

Artificial neural networks (ANNs) are powerful tools in the machine learning toolkit, used in a variety of ways to accomplish objectives such as image recognition, natural language processing, and game playing. ANNs learn in a similar way to other machine learning algorithms: by using training data. They are best suited to unstructured data where it’s difficult to understand how features relate to one another. This chapter covers the inspiration of ANNs; it also shows how the algorithm works and how ANNs are designed to solve different problems.

9.2 The Perceptron: A representation of a neuron

9.3 Defining artificial neural networks

9.4 Forward propagation: Using a trained ANN

9.5 Backpropagation: Training an ANN

9.5.1 Phase A: Setup

9.5.2 Phase B: Forward propagation

9.5.3 Phase C: Training

9.6 Options for activation functions

9.7 Designing artificial neural networks

9.7.1 Inputs and outputs

9.7.2 Hidden layers and nodes

9.7.3 Weights

9.7.4 Bias

9.7.5 Activation functions

9.7.6 Cost function

9.7.7 Learning rate

9.8 Expressing ANNs mathematically

9.8.1 The weighted sum as a dot product

9.8.2 The hidden layer as matrix multiplication

9.8.3 Adding the activation function

9.8.4 The output layer

9.8.5 The final neural network equation

9.8.6 The cost function

9.8.7 Expressing backpropagation mathematically

9.9 Artificial neural network types and use cases

9.9.1 Recurrent neural network

9.9.2 Convolutional neural network

9.9.3 Generative adversarial network

9.10 Summary of artificial neural networks