Prior to 2017, the majority of renditions of neural network models were coded in a batch scripting style. As AI researchers and experienced software engineers became increasingly involved in research and design, we started to see a shift in the coding of models that reflected software engineering principles for reuse and design patterns.
One of the earliest versions of using design patterns for neural network models was the use of a procedural style for reuse. A design pattern implies that there is a current best practice for constructing and coding a model that can be reapplied across a wide range of cases, such as image classification, object detection and tracking, facial recognition, image segmentation, super resolution, and style transfer.
So how did the introduction of design patterns aid in the advancement of CNNs (as well as in other architectures, such as transformers for NLP)? First, it aided other researchers in understanding and reproducing a model’s architecture. Decomposing a model into its reusable components, or patterns, provided a means for other practitioners to observe, understand, and then perform efficient device experiments.