We introduce two super-awesome algorithms! SVM SMO algorithm! And it's hard too!

KKT conditions

First look at how to select parameters. In the SMO algorithm, we select the parameters in turn:

It is a complicated and efficient method to select the sample point which violates the most serious condition of KKT, and to choose the second parameter with its corresponding parameter as the first parameter, but for a simple implementation, the second parameter, even if randomly selected, is not

You can visually understand the many concepts mentioned here in a single graph:

(The painting is a bit messy, forgive me ...) ）

There is a black circle outside of the figure is actually the legendary "support vector", its definition will be given at the end of the text

So how exactly should we characterize "violating KKT conditions"? Intuitively, we can have such a simple and effective definition:

Thanks to Numpy's powerful Fancy indexing, the implementation of the above 0 is very good to write (??? ）：

As for the second step of the SMO algorithm, as mentioned above, its essence is a constrained two-time planning, although the solution process may be more frustrating, but in fact, not very difficult. Specific steps will be placed at the end of the text, here is temporarily pressed

Effects of SMO

Still look at the training process on the Helix data set first:

It's a little tangled, but it's good.

Let's look at the performance of the mushroom dataset. For this data set alone, the naïve SVM we implement is almost identical to the SVM representation in Sklearn (when using the RBF kernel), the more representative training curve is shown:

It's also an intuitive way to meet SMO, a training method that only takes two parameters at a time to update.

(although these are in Python SVM (ii) · LINEARSVM introduced, but for consistency, here is again introduced again)

Thus, the optimal solution naturally needs to satisfy this condition:

At this point, the introduction of SVM algorithm is almost over. We introduce the "maximal gradient method", the MBGD (Batch gradient descent) method, the kernel method and the SMO algorithm from the perceptual machine, which is a bit of a taste, but it covers something ... It's probably quite a lot.

Hope that the audience can enjoy the master!

Incoming groups: 125240963 to get dozens of sets of PDFs OH

We introduce two super-awesome algorithms! SVM SMO algorithm! And it's hard too!