Tai Lin Xuan Tian Machine learning course note----machine learning and PLA algorithm

A probe into machine learning

1. What is machine learning
Learning refers to the skill that a person refines in the course of observing things, rather than learning, machine learning refers to the ability of a computer to gain some experience (i.e. a mathematical model) in a pile of data by observing it, thereby improving the performance (measurable) of certain aspects (such as the accuracy of the recommended system).
2, machine learning conditions of use
Need to have rules to learn
Have the data prepared beforehand
Programming is hard to do
3, machine learning elements of the composition
Input X output Y target function F:x->y data (training Set): d={(X1,y1), (x2,y2),.... (Xn,yn)} hypothesis (skill): g:x->y
4. How to use machine learning

Through the two inputs: training data set and hypothesis set (), using appropriate learning algorithm, finally get the final hypothesis model G as far as possible with the objective function f to coincide.

PLA algorithm

Course Teaching Ideas:

• First case, whether to issue credit cards to customers, the introduction of PLA algorithm
• Mathematical abstraction of a problem to get the objective function
• Explain the learning process of PLA in detail
• Prove whether PLA is convergent
• Point out the advantages and disadvantages of PLA and introduce PA algorithm

PLA is a linear classifier that accepts or rejects a given set of results
For credit card application questions, mathematical abstractions and models are as follows:

The first expression is the mathematical abstraction of the credit card application problem, that is, given the customer's eigenvector, calculates the weight score, sets a threshold (i.e. the threshold), and if the weighted sum score of the eigenvector exceeds the threshold, the credit card application is accepted, and conversely, the refusal.
The second expression is the resulting abstract mathematical model, set the mathematical symbol function h (x), the feature vector weighted sum score and the threshold is poor, if greater than 0, then accept, less than 0, then reject, and finally ignore 0
The model is written in the form of dot product:

The next step is the PLA's learning process:

The algorithm can be summarized as simple, by constantly correcting the weight vector W finally find a line that can completely separate the two kinds of results, that is, the hypothesis model G, the model correction method is:
With the rule of multiplying vectors, if we want the result to be positive, and W is the opposite of X, we need to make w close to X, so that WT adds a positive number, and if we want the result to be negative, and if WT and x happen to be close, then we need to adjust w so that W is away from X.
There is a problem here, by constantly adjusting whether W will eventually find a WT can completely separate the two types of results, that is, whether the PLA algorithm will terminate, whether it is convergent.
The conditions for the termination of PLA are:

The condition that satisfies this question is whether the existing data is linear, the course does not prove this condition, but directly proves that: linear separable d<=>exists Perfect Wf

To prove whether the WF and WT are close, the two vectors do the inner product, the greater the value of the inner product, indicating that the closer the two vectors or the longer the length of the vector, the following to deal with the length of the vector.

Using the nature of the PLA's "Fault only Update", in the case of making mistakes, through the above deduction, the final conclusion is that the square of WT length increases the square of xn longest length after each update.
Using the conclusion of the first proof, the derivation process is as follows:

The above is known as three conditions, there are two points to be explained:
1) Because the value of the vector angle is less than 1, the formula to prove less than 1, so we just need to prove that T is also bounded, it can be explained that the algorithm is convergent
2) The angle on the left side of the upper is getting smaller and closer, so we are using the correct method to solve the problem

Advantages and disadvantages of PLA:
Benefits: Simple, fast, and can be implemented in any dimension
Cons: 1) need to assume that the data is linear and can be divided
2) cannot determine how many times the algorithm will iterate
We use the greedy algorithm to solve this problem, its essence is a comparison, take a better result, to achieve the following:

Tai Lin Xuan Tian Teacher's teaching skills is very good, but these algorithms still have to deduce themselves to really understand. The first to write this kind of technology blog, a lot of content reference Happyangel blog, in this thanks.

Resources
1 Coursera Big Machine learning Cornerstone
2 Happyangel Blog--"machine learning definition and PLA algorithm"

Tai Lin Xuan Tian Machine learning course note----machine learning and PLA algorithm