Model complexity describes how much structure or flexibility a model can use to represent data. More parameters can improve fit to observed examples, but a highly flexible model may also fit noise that will not recur. Model selection therefore weighs fit against complexity, with generalization to unseen data as a central concern. section 3.7 section 13.2.1
The trade-off appears in different forms:
- Likelihood ratio tests, AIC, and BIC help compare plausible models by balancing fit against complexity. section 3.7
- A soft-margin support vector machine permits some margin violations or misclassifications so unusual observations do not dictate the boundary. section 8.0
- More neural-network parameters expand representational capacity, but capacity alone does not determine what the model learns; that also depends on training data and compute. section 13.2.1
The usual warning that greater flexibility can increase overfitting does not imply a universal complexity threshold. In the studied range, larger language models trained with sufficient data and compute continued to reduce held-out loss as parameter counts grew, following a power-law pattern. section 13.2.1



