Compositional generalization

As explained in Deep Learning with R, Third Edition

Compositional generalization is the ability to combine learned components to handle novel combinations of functions or situations.

Compositional generalization is the ability to combine learned components into new programs for situations that were not encountered during training. Current systems can often fetch and reuse memorized functions, yet struggle to compose them on the fly because their learned functions are not sufficiently abstract or modular. section 19.4.6

From one reusable function to a new program

Suppose an engineer packages HTTP-query handling in R as an abstract library. A later engineer can search for that component, download it, and combine it with other components in a different project. A compositional system would work similarly: it would select reusable algorithmic or geometric primitives, assemble them into a task-specific model, and adapt the resulting program to a new situation. section 19.4.6

A compositional approach depends on:

• Robust program components modeled on reusable functions and classes. section 19.4.6
• A library of high-level subroutines accumulated from many solved problems. section 19.4.6
• On-the-fly recombination that synthesizes programs suited to the current task. section 19.4.6

Figure 19.16 A meta-learner capable of quickly developing task-specific models using reusable primitives (both algorithmic and geometric), thus achieving extreme generalization

Shows a meta-learner developing task-specific models from reusable algorithmic and geometric primitives. section 19.4.6

How Deep Learning with R, Third Edition covers it

How deep it goes

Describes the limitation, rationale, and future architecture, but stops short of implementation steps or formal derivations.

Examples

Uses HTTP-query libraries in R, reusable functions and classes, and a meta-learner assembling algorithmic and geometric primitives.

What it assumes

Assumes familiarity with learned functions, programs, and modular components while motivating their role through software libraries.

What you can do after

Evaluate how reusable components and program synthesis could support adaptation beyond memorized patterns.

How it connects

Connects compositional generalization with function composition, program synthesis, meta-learning, modularity, and global reusable libraries.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Related concepts

• ARC-AGI-2measured by · ARC-AGI-2 is a benchmark dataset of abstract reasoning tasks designed to evaluate fluid reasoning and generalization to novel problems.
• language modeladdressed by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• compositionequivalent to · Composition is a technique for combining simpler functions, components, or operations into a larger structure or behavior.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.