The learning experience of statistical learning method (Hangyuan Li) (i.)

A blink of an eye, from the beginning of contact with machine learning, to now be trivial about, have to put down Hangyuan Li Teacher's "statistical learning method", nearly five months. For five months, the first one months were the happiest of my time, and it was wonderful to enjoy all the thinking that the statistical learning method brings.
By the end of the press, I probably looked at 1/4, considering the total-sub-structure of the book, the next reading task is to learn more model methods, than to deepen the understanding of statistical learning methods. I would like to state that this book will be read after the year, and will refer to other learning materials to systematically study several more classical statistical learning models.
What is the difference between statistical learning and machine learning? In fact, statistical learning is also called statistical machine learning, from the name can be seen, statistical learning from the perspective of statistical disciplines, using machines (programming algorithms) to predict data, machine learning is almost equal to statistical learning, machine learning is not dependent on statistical methods. No information has been found to confirm the present. My personal understanding is that the term statistical learning is more applicable to the subject theory, machine learning is biased to engineering practice.
The beginning of the book Ming Yi, in the first chapter 1.3 brings out the three elements of statistical learning: Model + strategy + algorithm, the following study of each learning method, the basic is from these three elements to unfold. In particular: A model: a relationship or a rule. In the book it is said that the "conditional probability distribution or decision function to be studied" is linear or non-linear, whether it is a probabilistic or not probabilistic model, and all possible (some type of model) constitutes the hypothetical space of the model; strategy: how to learn. Take Alphago For example, its learning strategy is the current value of each move is the best, that is, every child falling, is towards the goal of winning. The book is written as "what kind of quasi-test to learn or choose the best model". In general, when we are training data, the difference between the predicted value and the real value is compared, the smaller the better, this is the strategy; algorithm: The optimal model has been selected, how to calculate the parameters inside. At this time, the algorithm came into play, "statistical learning algorithm to solve the optimization problem algorithm." Algorithm is very important, the algorithm is fast and slow, there is no right, in the statistical study, this is one of the big head.
There is also a situation where we use the model is very complex, the algorithm is very sophisticated, so that the training of data perfect match our model, but this model to predict the unknown data, the performance is very poor, this phenomenon is called over-fitting. Over-fitting phenomena often occur when we improve the accuracy of training models, which can be avoided by regularization and cross-validation, which is described in detail in the book.
How to evaluate a statistical learning method to learn the model, the ability to predict the unknown data. First, we call this ability to be generalized and quantify it with generalization errors.
Next is the classification of the model, the book refers to the supervision of learning (the focus of this book) can be divided into generating methods and discriminant methods, the learning model for the generation of models and discriminant models. The generation model represents the relationship between a given input and output, and the discriminant model is concerned with the given input and what kind of output should be predicted. This one can read and understand the difference carefully.
The end of the first chapter is the types of problems that we can solve through statistical learning, including classification, labeling, regression and so on. Space is limited, to really understand the concepts mentioned above, you have to work in two directions: 1. Familiar with common learning methods 2. Practice by programming examples
encourage each other