A parametric curve determines points through a finite set of parameters and one or more independent variables. In deep learning, models are described as large parametric curves fitted to datasets. section 19.1.1
This framing explains both the practical strength and the limitations of deep learning: parameterized models are easy to train and scale, but their fitted information remains fixed after training. section 19.1.1
Treating a parametric curve as a stored database highlights three consequences:
- Stores information captured from training data rather than continuously adapting to new situations. section 19.1.1
- Retrieves or generates patterns resembling those encountered during training. section 19.1.1
- Can fail when inference data falls outside the training distribution. section 19.1.1
Leopard-print sofa
A model trained on ImageNet may classify a leopard-print sofa as a leopard because its fitted patterns do not adequately represent sofas as a separate situation. section 19.1.1
Parametric curves therefore represent one way of storing abstractions, while challenges such as ARC-AGI motivate alternatives involving search and symbolic manipulation. section 19.4
