Support Vector Machine notes (5) regularization and SMO

So far, the SVM is described as being in a low-dimensional, or mapped to a high-dimensional post-linear can be divided, but for some outliers situation, we get the super plane is not necessarily the best, as in the image below, this outliers significantly affect the division of the hyper-plane:

In order for this algorithm to become less sensitive to outliers, we added regularization to the optimization problem:

Compared with previous results, the only difference is that Alpha has an upper limit of C, so that a svm that can tolerate outliers is produced. Below we will introduce the next sequential minimal optimization (SMO) algorithm, which is used to optimize the dual problem. To explain the SMO algorithm, we first introduce coordinate ascent, which briefly describes the problems that need to be optimized:

In fact, fixed other values unchanged, change the alpha (i) to the maximum W value, the following is the graph of the algorithm:

The following is an official explanation of SMO, recalling the dual problem we have previously optimized:

However, if we want to use the above-mentioned coordinate ascent, we find that if the fixed alpha (2--m) optimization alpha (1) is based on (19) It has been determined that no optimization is needed, that is, it is not possible to optimize only one volume, so we optimize two variables here:

The diagram shows the following:

According to the limiting condition of (18) Alpha (1), alpha (2) must be within the range of [0,c]*[0,c], the maximum value of this graph I think is very clear is the line and the intersection of the graph, the problem is solved.