some time ago on the Internet to see the Coursera Open Classroom Big Machine learning Cornerstone Course, more comprehensive and clear machine learning needs of the basic knowledge, theoretical basis to explain. There are several more important concepts and ideas in foundation, first review, and then open the follow-up techniques to learn and summarize the course.
1. VC Dimension (VC dimension, very important concept)Ability to shutter the upper limit of two classification problems. is also a tool for measuring the complexity of a model (similar to the concept of degrees of freedom). The reason why this concept is more important is that it can explain why machines can learn.
1), in the probability of statistics commonly used means: using sample to estimate the whole, machine learning is the same, that is, by sampling the sample to learn, can be used to estimate the out of sample, processing, prediction, classification and so on. The so-called learning is from a bunch of hypothesis (set), the use of sample, through the Learning algorithm race to select the appropriate hypothesis-g process.
2) Plug-in criteria are usually various types of error (0/1error,square error ... ), these errors are used to adjust the W weights and finally get the hypothesis (g) of the smaller error (in sample).
3) This g is only good in the in sample, actually in the sample on the good performance and no eggs, because if you are only processing in the sample data, there is no need for machine learning, the reason for using machine learning, because it is impossible to get all the data, You can only sample part of the sample. So the best g should be done on the out of sample. Because, we can't measure error in sample, so the best way is to establish the connection between the error in sample and the error out of sample, can there be a upper bound to measure the relationship between the two? The answer is yes, that's hoeffding's inequality.
4) Hoeffding inequality illustrates a problem, if hypothesis set hypothesis can shutter many kinds (that is, VC dimension Very Large), it will cause this error in the sample and error out of Sample is quite different, which means that the model is very complex. This error in sample you can do very little, but the error out of the sample will be very large.
5) VC d/A model complexity high =>error in sample small = model is not smooth enough =>generalization ability weak =>error out of the sample big =>overfitting= The > model has no OVA.
2. Generalization (generalization ability)1) measure the performance of the model on the out of sample;
2) Usually the smoother the curve, the more generalization ability, but the error in the sample may be larger, underfitting, the curve is also complex, error in the sample can be done less, but the less generalization ability, overfitting;
3. Regularization (regularization)1) To control the complexity of the model, so as to achieve error in the sample and error out of the sample approximation, that is to make both a good precision, but also a good generalization ability;
2) different regularizer correspond to different regression methods: L1,l2,... is actually a punitive measure. Used to weigh the good error and generalization ability;
4. Validation
A method used to measure the generalization ability of machine learning. Because machine learning is hypothesis to be processed on the out of sample, not on the in sample. So, a means to evaluate whether machine learning is in place is from validation. The general practice is to separate the prior data set into training set and validation set, use the training set for hypothesis learning, use the validation set to determine the learning termination condition, and give the Learning Hypothesis performance index. However, if you separate the datasets, fewer samples are used for training. We know that the number of samples in the training set N is a significant factor in machine learning to prevent overfitting. If the model is more complex, it is often necessary to increase the number of training samples to overcome the risk of overfitting caused by the complexity of the model. Neural networks, for example, are a typical example. So it is better to be able to not reduce the number of samples in the training set, but also to validation. This presents the validation and n-folder validation of leave one out.
In addition to the above four concepts and ideas that I think are extremely important, there are some main contents such as: the excessive use of VC dimension,noise and limited data size N, several methods to solve the overfitting, overfitting Tip: Validation (cross Validation,leave one out validation, N-folder valiation ... ), data hinting, data cleaning/pruning, regularization, start from the simple model, and so on. This is no longer a summary.
*********************************
2015-8-16
Less art
Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.
Machine learning fundamentals and concepts for the foundation course of machine learning in Tai-Tai
