Machine Learning Basics (vi) Support vector machines

The last section basically completes the theory of SVM to tear down, to find the goal of maximizing the interval to solve the problem of solving Lagrange multiplier Alpha, the Alpha can solve the weighted W of SVM, and the maximum interval distance is obtained with the weight, but in fact we have a hypothesis in the last section: Is that the training set is linear and can be divided so that the alpha is in [0,infinite]. But what if the data is not linearly divided? At this point we will allow a part of the sample can cross the classifier, so that the optimization of the objective function can be unchanged, as long as the introduction of the relaxation variable, it represents the wrong classification of the cost of the sample point, classification is correct when it equals 0, when the classification error, Where TN represents the true label of the sample-1 or 1, review the previous section, we fixed the distance of the support vector to the classifier at 1, so the distance between the support vectors of the two classes is definitely greater than 1, and when the classification error is definitely greater than 1, as shown in (Figure V) (where the formula and icon numbers are connected to the previous section).

(Figure V)

This has the cost of the wrong classification, and we add this error classification cost to the objective function of the last section (Formula Four), and get the form of (Formula eight):

(Formula Eight)

Repeat the previous section of the Lagrange multiplier method step, obtained (Formula Nine):

(Formula Nine)

More than one UN multiplier, of course our job is to continue to solve this objective function, continue to repeat the steps in the previous section, derivation (Formula 10):

(Formula 10)

And because Alpha is greater than 0, and UN is greater than 0, so 0<alpha<c, for the sake of clarity, we send out the Kkt condition (the third type of optimization problem in the last section), and note that UN is greater than or equal to 0: