A black-box model produces predictions without making its internal decision process readily understandable or inspectable to its users. Machine learning often accepts this trade-off when predictive performance matters more than explaining how the model works. section 1.5
Black-box modeling is especially visible when:
- Models are trained on labeled data and evaluated on labeled data they have not seen before. section 1.5
- Complex models such as random forests and neural networks make explanations harder as the number of input variables grows. section 8.6
- Strong test performance does not guarantee safe real-world behavior, especially when biased decisions affect groups of people. section 1.5
A model that predicts well but explains little
A simple logistic regression can expose its workings, so it need not be treated as a black box. As the same classification task expands toward random forests or neural networks, evaluation shifts toward repeated train/test performance while the model's internal reasoning becomes less accessible. section 8.6
