Depth
Neural-network depth is the number of successive computational layers in a neural-network architecture.
Grokking Machine Learning
The number of neural-network layers excluding the input layer.
- How it is used
- Defines a neural network’s architecture by reporting how many processing layers follow its input layer.
- How deep it goes
- Explains the counting rule and distinguishes hidden, output, input, and bias nodes without deriving network internals.
- Examples
- Uses a network sized 4-5-3-1 and diagrams showing increasingly complex classifier boundaries across hidden layers.
- What you can do after
- Evaluate neural-network diagrams by identifying their layers and counting depth correctly.
Deep Learning with PyTorch, Second Edition
The extent to which a neural network has many stacked layers or sequential operations, increasing representational complexity.
- How it is used
- Uses depth to increase representational complexity and process hierarchical visual information.
- How deep it goes
- Explains depth through intuition, sequential operations, gradient difficulties, and PyTorch architectures with up to 100 residual blocks.
- Examples
- Uses person-recognition hierarchies, three-layer NetDepth and NetRes models, residual blocks, and a 100-block network.
- What you can do after
- Build deeper convolutional networks by stacking layers or repeating residual blocks in PyTorch.
Read how Deep Learning with PyTorch, Second Edition explains depth →
Grokking AI Algorithms, Second Edition
The number of layers is the count of successive computational layers in a neural-network architecture.
- How it is used
- Uses depth to configure neural-network architectures and represent increasingly complex relationships or reasoning chains.
- How deep it goes
- Explains depth through hidden-layer abstractions and Transformer steps, but defers formal optimization or theoretical analysis.
- Examples
- Uses simple classification boundaries, multilayer ANN architectures, and toy-versus-real-world Transformer configurations.
- What you can do after
- Evaluate candidate layer counts through architecture design, experimentation, and iterative improvement.
Read how Grokking AI Algorithms, Second Edition explains depth →
Timeless Algorithms, The Foundational Ideas
Network depth is the number of successive computational layers in a neural network.
- How it is used
- Explains how neural networks compose representations to capture increasingly complex relationships.
- How deep it goes
- Defines depth as hidden-layer count and explains progressive transformations, while treating layer-specific roles as illustrative rather than fixed.
- Examples
- Uses pedestrian recognition to trace visual patterns from pixels through contours, shapes, and classification evidence.
- What you can do after
- Understand how depth supports compositional capacity and why greater capacity alone does not guarantee generalization.
Read how Timeless Algorithms, The Foundational Ideas explains depth →
Machines that Think
Model depth is the number of successive computational layers or transformation stages in a model.
- How it is used
- Explains depth as a way to build and combine representations across successive model layers.
- How deep it goes
- Defines depth through successive transformations and connects it to representation building and training instability, without formal measurement detail.
- Examples
- Uses image features that progress from edges to shapes, sentence relationships around “rises,” and tasks such as face recognition and sentiment detection.
- What you can do after
- Evaluate how greater depth can expand representational capacity while increasing training challenges.
Graph Neural Networks in Action
The number of computational layers in a neural network, especially the number of graph-processing layers in a graph neural network.
- How it is used
- Uses depth to tune GNN receptive fields and refine GraphSAGE model performance.
- How deep it goes
- Defines layers as graph hops, compares 2, 10, and 50 layers, and explains under-reaching, over-smoothing, overfitting, and complexity.
- Examples
- Compares 2-, 10-, and 50-layer training curves and implements two-layer GraphSAGE models with configurable aggregation and dropout.
- What you can do after
- Evaluate GNN depth by comparing training and validation behavior while configuring GraphSAGE layers for experiments.
Also covered in
- Bayesian Optimization in Action
- Grokking Machine Learning, Second EditionIn early access (MEAP)
- Rearchitecting LLMsIn early access (MEAP)
- Sutskever's List
What to know first
- machine learningMachine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- neural networkA neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- backpropagationBackpropagation is reverse-mode differentiation that propagates derivatives backward through a computational graph to compute gradients of an output or loss with respect to inputs and parameters.
Related concepts
- pipeline parallelismused by · Pipeline parallelism is a form of model parallelism that divides computation into sequential stages assigned to different devices, allowing multiple inputs or microbatches to progress through the stages concurrently.
- neural networkconfigures · A neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
- hyperparameterkind of · A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
- over-smoothingproduces · Over-smoothing is a graph neural network failure mode in which repeated neighborhood aggregation or message passing makes node representations increasingly similar and difficult to distinguish.
- receptive fieldproduces · A receptive field is the set or region of input elements whose information can influence a particular neural-network unit, representation, or output.
- graph neural networkconfigures · A graph neural network is a neural-network model that learns representations or predictions from graph-structured data by combining information from connected entities.
Questions about depth
What is depth?
Neural-network depth is the number of successive computational layers in a neural-network architecture.
Which Manning books explain depth?
Grokking Machine Learning, Deep Learning with PyTorch, Second Edition, Grokking AI Algorithms, Second Edition, Timeless Algorithms, The Foundational Ideas, Machines that Think and Graph Neural Networks in Action.
What should I know before learning depth?
Manning books that teach depth usually introduce machine learning, neural network, attention and backpropagation first.
What is depth a kind of?
Depth is a kind of hyperparameter. A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
What is depth also called?
Depth is also called number of layers.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





