7 Decision Trees
He that climbs the tree has won the right to the fruit.
- Walter Scott, an 18th centenary Scottish poet.
In the last part, we explored similarity based method which was founded on the general assumption of target function’s uniform continuity; this assumption didn’t constraint our model to a specific functional form. However, in this part we start exploring a specific form of modeling assumptions, which is the form of a tree. In this chapter, we solve a problem of used cars price prediction (which is our first regression problem to encounter so far) by assuming that our target function takes the form of a binary decision tree. Moreover, we’re going to learn:
- How to fill-in missing entries in a dataset using the rest of available entries.
- How to train and evaluate regression models using mean squared errors.
- How decision trees and random forests are built.
- How to use scikit-learn to train and use decision trees and random forests.
- What makes learning functions from data possible and what constraints it.
- What Bias-variance Trade-off is.