22 Training nonlinear classifiers with decision tree techniques

Thus far, we have investigated supervised learning techniques that rely on the geometry of data. This association between learning and geometry does not align with our everyday experiences. On a cognitive level, people do not learn through abstract spatial analysis; they learn by making logical inferences about the world. These inferences can then be shared with others. A toddler realizes that by throwing a fake tantrum, they can sometimes get an extra cookie. A parent realizes that indulging the toddler inadvertently leads to even more bad behavior. A student realizes that through preparation and study, they will usually do well on their exam. Such realizations are not particularly new; they are part of our collective social wisdom. Once a useful logical inference has been made, it can be shared with others for broader use. Such sharing is the basis of modern science. A scientist realizes that certain viral proteins make good targets for a drug. They publish their inferences in a journal, and that knowledge propagates across the entire scientific community. Eventually, a new antiviral drug is developed based on the scientific findings.

22.1 Automated learning of logical rules

22.1.1 Training a nested if/else model using two features

22.1.2 Deciding which feature to split on

22.1.3 Training if/else models with more than two features

22.2 Training decision tree classifiers using scikit-learn

22.2.1 Studying cancerous cells using feature importance

22.3 Decision tree classifier limitations

22.4 Improving performance using random forest classification

22.5 Training random forest classifiers using scikit-learn

Summary