Abstraction & Reasoning Corpus for Artificial General Intelligence
The Abstraction and Reasoning Corpus for Artificial General Intelligence is a benchmark of visual reasoning tasks designed to assess abstract rule induction and generalization from examples.
Sutskever's List
ARC-AGI is a benchmark of small grid-based puzzles that evaluates abstract rule induction from a few examples.
- How it is used
- Uses small grid puzzles to evaluate abstract rule induction through standardized empirical comparison.
- How deep it goes
- Frames the symbolic-connectionist reasoning debate but does not detail ARC task-solving algorithms or implementations.
- Examples
- Uses small grid-based puzzles requiring a rule inferred from a few training examples.
- What you can do after
- Evaluate claims about reasoning by comparing systems on shared abstraction tasks and measured baselines.
Deep Learning with Python, Third Edition
A benchmark dataset designed to measure generalization and abstract reasoning on tasks that are easy for humans but challenging for standard AI systems.
- How it is used
- Measures whether AI systems generalize abstract rules to unfamiliar grid tasks instead of relying on memorization.
- How deep it goes
- Defines the task format and evaluation rationale, then contrasts memorization with test-time adaptation without specifying grid-solving algorithms.
- Examples
- Uses input-output grid pairs, new grid predictions, three-attempt solving, and reported LLM benchmark scores as concrete examples.
- What you can do after
- Evaluate whether a system exhibits flexible abstraction and diagnose when scaling or memorization fails to produce fluid reasoning.
Deep Learning with R, Third Edition
ARC-AGI-1 is the original version of the ARC-AGI benchmark, consisting of abstract reasoning tasks intended to test fluid intelligence.
- How it is used
- Measures fluid intelligence by testing abstract reasoning and generalization on previously unseen grid-transformation tasks.
- How deep it goes
- Defines the task format and rationale, but stops short of specifying individual grid transformations or solving algorithms.
- Examples
- Uses paired input–output grids, an unseen input grid, three attempts, and Figure 19.9 as its concrete task illustration.
- What you can do after
- Evaluate reasoning systems by comparing their ability to infer novel grid transformations rather than memorize known strategies.
Related concepts
- generalizationmeasures · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
- ARC-AGI-2contrasts with · ARC-AGI-2 is a benchmark dataset of abstract reasoning tasks designed to evaluate fluid reasoning and generalization to novel problems.
- fluid intelligencemeasures · Fluid intelligence is the ability to solve novel problems and adapt reasoning to unfamiliar situations without relying solely on previously learned knowledge.
- memorizationcontrasts with · Memorization is the retention of specific information or examples in a form that enables their later recall or reproduction.
- language modelcontrasts with · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.