The Modularity-Hierarchy-Reuse (MHR) formula simplifies complex systems by dividing them into modules, arranging those modules hierarchically, and reusing them where appropriate. In deep learning, layers act as modules, networks form hierarchies, and shared operations reduce the search space that gradient descent must explore. section 9.1 section 9.0
Apply the formula through three architectural choices:
- Modularity: package computation into layers, blocks, or modules with distinct roles. section 9.1
- Hierarchy: organize modules into deeper structures, such as stacked layers or feature pyramids. section 9.1
- Reuse: apply the same module or operation in multiple places instead of reimplementing it; convolutions reuse information across spatial locations. section 9.1
Xception entry flow
Xception illustrates MHR with repeated SeparableConv–SeparableConv–MaxPooling blocks. The network also forms a hierarchy by gradually shrinking feature maps while increasing their depth, from 299 × 299 × 3 to 19 × 19 × 728. section 9.1
Figure 9.1 Complex systems follow a hierarchical structure and are organized into distinct modules, which are reused multiple times (such as your 4 limbs, which are all variants of the same blueprint, or your 20 digits).
Use the figure to see how complex systems combine distinct modules, hierarchical organization, and repeated blueprints. section 9.1
