Life often requires us to estimate the chances of occurrence of some event or to make a decision in the face of uncertainty. Probability and statistics is the common box of tools to use under these circumstances. In this chapter we will study this from a machine learning point of view. As usual, we will emphasize on the geometrical view of multivariate statistics. This chapter is strongly connected to the previous chapter 4. The reader is encouraged to read these two chapters in a back and forth fashion.
In machine learning, we take large feature vectors as inputs. and ascribe them to one or more of pre-defined classes. Such a machine is called a classifier.
As stated earlier, we can view the feature vectors as points in a high dimensional space. Now suppose, with each point in the input space we associate the probabilities of belonging to each of the possible classes. Then, given any input we will simply pick the class with the highest probability. In effect, we have a classifier. Thus, we are modelling the probability distributions of the classes over input space.