SVM entry (2) linear classifier Part 1

Linear Classifier(In a sense, it can also be calledPerception Machine) Is the simplest and most effective classifier form. in a linear classifier, we can see the idea of SVM formation and come into contact with many core concepts of SVM. here is a small example of the classification problem of only two types of samples in a two-dimensional space.

650) This. width = 650; "src =" http://images.blogjava.net/blogjava_net/zhenandaci/WindowsLiveWriter/hi_8CA8/clip_image002_2.gif "/>

C1 and c2 are the two types to be distinguished, as shown in the sample of them in a two-dimensional plane. The straight line in the middle isClassification functionsWhich can completely separate the two types of samples. Generally, if a linear function can completely separate samples correctly, the data is calledLinear DifferentiationOtherwiseNon-linear Differentiation.

What isLinear FunctionsWhat about it? In a one-dimensional space is a point, in a two-dimensional space is a straight line, in a three-dimensional space is a plane, as you can imagine, if you do not focus on the dimension of the space, this linear function also has a uniform name --Hyper plane)!
In fact, a linear function isReal-value functions(That is, the value of the function is a consecutive real number), and our classification questions (for example, the binary classification question here -- to answer the question of whether a sample belongs to or does not belong to a category) output values that need to be discrete. For example, 1 indicates that a sample belongs to the Class C1, and 0 indicates that it does not belong to (if it does not belong to C1, it means it belongs to C2 ), in this case, you only need to simply attach a threshold value to the real-value function. You can determine the category attribution by calculating whether the value obtained by the classification function is greater than or less than the threshold value. For example, if we have a linear function g (x) = wx + B, we can take the threshold value as 0, so that when there is a sample XI that needs to be identified, we will look at G (XI). If G (xi)> 0, it is classified as Class C1. If G (xi) <0, it is classified as Class C2 (when it is equal to, we refuse to judge, huh, huh ). In this case, it is equivalent to attaching a symbol function SGN () to function g (x), that is, f (x) = SGN [g (x)] is our real discriminant function.

Note three points for the expression g (x) = wx + B:

1. X in the formula is not a horizontal axis in a two-dimensional coordinate system, but a vector representation of the sample. For example, if the coordinate of a sample point is (), XT = ), instead of X = 3 (generally, vectors are column vectors, so transpose is added when expressed in the form of row vectors ).

2. This form is not limited to two-dimensional conditions. We can still use this expression in the n-dimensional space, but w in the formula becomes an n-dimensional vector (in the two-dimensional example, W is a two-dimensional vector. In order to make it easy and concise to express it, the column vector and its transpose are no different below, so smart readers can see it at a Glance );

3.g (x) is not the expression of the middle line. The expression of the middle line is g (x) = 0, that is, wx + B = 0, we also call this functionClassification Surface.
In fact, it is easy to see that the demarcation line in the middle is not the only one. We will rotate it a bit, as long as the two types of data are not divided incorrectly, the above results can still be achieved, you can also translate it a little. This involves a problem. Which function is better when there are multiple classification functions for the same problem? Obviously, you must first find an indicator to quantify the "good" degree. Generally, it is called"Classification Interval. Next, let's take a closer look at the classification interval and make up the relevant mathematical knowledge.

From: http://blog.sina.com.cn/s/blog_5f853eb10100qbb9.html

This article is from the squirrel blog, please be sure to keep this source http://apinetree.blog.51cto.com/714152/1560190

SVM entry (2) linear classifier Part 1