Chapter 6. Support vector machines

 

This chapter covers

  • Introducing support vector machines
  • Using the SMO algorithm for optimization
  • Using kernels to “transform” data
  • Comparing support vector machines with other classifiers

I’ve seen more than one book follow this pattern when discussing support vector machines (SVMs): “Here’s a little theory. Now SVMs are too hard for you. Just download libsvm and use that.” I’m not going to follow that pattern. I think if you just read a little bit of the theory and then look at production C++ SVM code, you’re going to have trouble understanding it. But if we strip out the production code and the speed improvements, the code becomes manageable, perhaps understandable.

Support vector machines are considered by some people to be the best stock classifier. By stock, I mean not modified. This means you can take the classifier in its basic form and run it on the data, and the results will have low error rates. Support vector machines make good decisions for data points that are outside the training set.

6.1. Separating data with the maximum margin

6.2. Finding the maximum margin

6.3. Efficient optimization with the SMO algorithm

6.4. Speeding up optimization with the full Platt SMO

6.5. Using kernels for more complex data

6.6. Example: revisiting handwriting classification

6.7. Summary