The latest revolution in deep learning is at the macro level rather than the micro level, with the introduction of an approach that I coined while working at Google Cloud AI as model amalgamation. In this approach, models are broken into composable units that share and adapt components to achieve different objectives with the same initial data. The components are interconnected in a variety of connectivity patterns, in which each component learns communication interfaces between the models through design, without the necessity of a backend application.
In addition, model amalgamation can be used to train Internet of Things (IoT) devices for data enrichment, turning IoT sensors from static to dynamically learning devices—a technique called model fusion. Amalgamation is providing the means for putting AI into production at a scale and operational complexity not conceivable in 2017, when the push into production first started to emerge.