An inductive bias is an explicit or implicit assumption that makes an inference algorithm prefer some predictions or explanations over others. It constrains how a model generalizes beyond observed data. Statistical inference uses assumptions of this kind; causal inference adds causal assumptions to distinguish cause and effect from correlation. section 5.3.1 section 1.1
Inductive biases enter machine-learning workflows through:
- Model architectures and objectives: convolutions and max pooling encode translation invariance, treating an object as stable when its image location changes. section 5.3.1
- Causal models: a causal directed acyclic graph encodes assumptions about the data-generating process and can guide learning. section 5.3.1
- Training data, procedures, hyperparameters, prompts, and benchmark data: causal information may need to be deliberately embedded when evaluating causal reasoning. section 10.0
Inferring study effort from test scores
Suppose C is study effort and O is a test score. Knowing how effort causes scores, P(O|C), does not determine how likely different effort levels are, P(C). A low score can therefore support an inference about effort only after assumptions about score frequencies or the mechanism are supplied. Independence of mechanism suggests that knowing P(O) may help constrain the inference of P(C|O). section 5.3.1
For causal machine learning, more data and a suitable architecture do not automatically solve the problem. Causal identification determines when observational or experimental data support a causal or counterfactual inference and reveals when additional data cannot compensate for missing causal assumptions. The y0 library implements graph-based identification algorithms. section 10.0
