The cornerstone of machine learning-Lin Xuan-Tian Five lecture notes

Last class, we mainly introduced the feasibility of machine learning. First of all, the NFL theorem shows that machine learning is seemingly unworkable. However, after the introduction of statistical knowledge, if the sample data is large enough, and the number of hypothesis is limited, then machine learning is generally feasible. This lesson will discuss the core issues of machine learning, and strictly prove why machines can learn. Starting from the last issue of class, that is, when the number of hypothesis is infinite, the feasibility of machine learning is still tenable. I. Recap and Preview

Let's take a look at the statistics based machine learning Flowchart:

In this flowchart, samples of the training sample D and the final Test H come from the same data distribution, which is a prerequisite for the machine to learn. In addition, the training sample d should be large enough, and the number of hypothesis set is limited, so that according to Hofding inequalities, will not appear bad Data, to ensure that ein≈eout, that is, a good generalization ability. At the same time, by training, we get the Ein of the smallest h, as the final moment of the model g,g close to the objective function.

Here, we summarize the main contents of the first four sessions: the first class, we introduced the definition of machine learning, the goal is to find the best moment G, so that g≈f, to ensure that eout (g) ≈0; the second class, we introduced how to make ein≈0, can use the PLA, Pocket algorithm to achieve; the third lesson , we introduced the classification of machine learning, our training samples are batch data (batch), processing supervised (supervised) two-yuan classification (binary classification) problem; Fourth class, we introduced the feasibility of machine learning, through statistical knowledge, put Ein (g) is associated with eout (g), proving that, under certain conditions,