Nonlinear transformation of "Machine Learning Foundation"

Introduction

In the classification problem described earlier, the model of the classification involved is linear, and in the section of nonlinear transformations, we extend the model to the non-linear situation to classify the data.

Two times hypothesis (quadratic hypotheses)

We see the above example, in the left image, the circle and the Red Fork data cannot be divided into two categories in a straight line, so this example, we can actually use a large circle to classify the data, so now we consider the assumption that hsep (x) is a circle that crosses the origin. This revelation allows us to solve a wider range of problems with a systematic approach to the linear classification methods we learned before.
Take the φ hypothesis of the circle above as an example, H (x) =sign (0.6 1 + (-1) x1^2 + (-1) x2^2). We make w0=0.6,w1=-1,w2=-1, and z0=1,z1=x1^2,z2=x2^2. In this way we can change the previous h (X) to sign (WT * z), the familiar form of the linear classification we learned before, and the only thing we do differently is to convert the space of the previous x into the new Z space. The process of converting every point in the X space to each point in the Z space is called feature conversion (Feature Transform). It is worth mentioning that, in the X-space with two-time assumptions can be divided into a linear in the Z space can be divided, but the reverse is not possible, because the line in the Z space is not necessarily in the X-space is a positive circle, there may be a hyperbolic curve, such as two times, So in z space, the linear line of data can be divided into lines that correspond to the specific curves of the X-space.

We can consider a more extensive two-time hypothesis, which is a hypothesis that allows the data to be linearly divided in the Z-space, where the conversion function is defined.

Nonlinear transformations (nonlinear Transform)

We can summarize this non-linear transformation step, that is, by φ (x) the point of the X-space into the point of the Z-space, and in the Z-space to get a linear hypothesis, and then revert to the original X-space to get a two-time hypothesis (the process of inverse operation does not necessarily exist).

In fact, this feature conversion is very important, for example, in the case of handwritten numeral classification, we transform the original pixel's characteristic data into more specific and physical meaning, and then solve the classification. This example is actually a linear classification in the new feature space, and the original pixel space is actually a non-linear hypothesis.

Cost of the nonlinear transformation (price of nonlinear Transform) calculation/storage cost (computation/storage prices)

Now let's consider a very general nonlinear transformation, which turns the two-time polynomial transformation into Q-times.

We use D to represent the dimensions in the z-space, and we need to get the different combinations of D-dimensions, which are O (q^d).
This number represents the computational complexity that we need to calculate the φ (x) transformation, the calculation of the parameter W (because of the time complexity of some training algorithms and the dimensions of the data), and the cost of storing W.

Modeling complexity (model complexity price)

We know that the parameters of this z-space model are 1+d, which is equivalent to the VC dimension of z space, so when Q becomes larger, the VC dimension becomes larger.

Generalization problem (generalization Issue)

We go back to machine learning is basically a balance between the compromise problem, if D (q), we can make ein very small, but this will lead to Ein and eout very different, when D (Q) small, can make Ein and eout difference small, but can not guarantee that the Ein is very small.

Reprint please indicate the author Jason Ding and its provenance
GitHub home page (http://jasonding1354.github.io/)
CSDN Blog (http://blog.csdn.net/jasonding1354)
Jane Book homepage (http://www.jianshu.com/users/2bd9b48f6ea8/latest_articles)



Nonlinear transformation of "Machine Learning Foundation"