Fully connected neural network

As explained in Grokking Machine Learning

A neural-network layer in which each unit receives input from all units in the preceding layer.

In a fully connected neural network, every unit in a layer receives input from every unit in the preceding layer. A small network can classify images after each image is flattened into a vector. section 10.3

Figure 10.21 The architecture that we will use to classify our dataset. It contains two hidden layers: one of 128 and one of 64 nodes. The activation function between them is a ReLU, and the final activation function is a softmax.

This classifier has two hidden layers, with 128 and 64 units, followed by a two-class output layer. section 10.3

Classifying handwritten digits

For MNIST, each 28-by-28 image is reshaped into a vector, and each label is encoded as a vector of 10 values. A network with 128- and 64-unit hidden layers predicts one of 10 digits. Its testing accuracy is 0.942; one shown error is a handwritten 3 classified as an 8. section 10.3

Flattening an image loses information about its layout. Fully connected networks can work well on simple image tasks, but more complex images call for other architectures, such as convolutional neural networks. section 10.5

How Grokking Machine Learning covers it

How deep it goes

Demonstrates network architectures, training, and testing evaluation, then notes where flattened-image inputs fall short.

Examples

Uses MNIST digit images, a circular binary dataset, and a 3 misclassified as an 8.

What it assumes

Prepares flattened image vectors and one-hot labels; does not specify assumed background.

What you can do after

Apply TensorFlow Keras to train a small classifier and evaluate its predictions on test data.

How it connects

Contrasts fully connected networks with convolutional neural networks and connects them to image recognition.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA dense neural network is a neural network architecture in which units in one layer are connected to all units in the adjacent layer.

Compare how 2 books explain fully connected neural network →

Related concepts

• N-BEATSused by · N-BEATS is a neural forecasting model organized as stacks of blocks for modeling time series without requiring explicit series decomposition or input transformations.
• deep learningkind of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• linear layerhas part · A linear layer is a neural-network layer that maps input features to outputs by a learned affine transformation, typically connecting each output to every input feature.
• recurrent neural networkcontrasts with · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
• feedforward networkequivalent to · A feedforward network is a neural network or subnetwork in which information flows from inputs toward outputs through successive computations without recurrent connections or cycles.
• sequencecontrasts with · A sequence is an ordered collection of elements whose positions determine their order of traversal or processing.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.