Classification of machine learning algorithms based on "machine Learning Basics"--on how to choose machine learning algorithms and applicable solutions

Introduction

The systematic learning machine learning course has benefited me a lot, and I think it is necessary to understand some basic problems, such as the category of machine learning algorithms.
Why do you say that? I admit that, as a beginner, may not be in the early stage of a learning object has a comprehensive and clear understanding and review, but, some key concepts have a preliminary and clearer understanding, help us to grasp the level of understanding of the problem, plainly, is to help us have the purpose to learn knowledge, with problems to learn, Filled with the power to solve the problem, I think this approach is beneficial and benign.
Before, I encountered a lot of this problem, probably because of the problem analysis is not enough, in the search for solutions to the problem of the method or model, may be overwhelmed. This can be caused by two points: 1, the foundation is not deep, do not understand the most commonly used algorithm model; 2. In the course of learning, the lack of understanding of the practical problems applied to the model and the lack of experience in applying the model to practical problems.
Therefore, in the course of learning, we should pay special attention not only to the essence of the algorithm, but also to know the application of the algorithm, the applicable conditions and limitations. If only to explore the principle, and do not understand the actual application, can only be a nerd, will only talk on paper, if only want to use, not to delve into the essence of the algorithm, but also free from the core technology boundaries, can not really understand. Only by combining theory and practice can we achieve maximum learning effect. From this path, we must persevere.

Input space, feature space, and output space

The input and output space is actually the set of all possible values for input and output. The input and output spaces can be either a set of finite elements or the entire European space. The input space and output space can be either a space or a different space; Typically, the output space is much smaller than the input space.
Feature space is the space in which all eigenvectors exist. Each dimension of the feature space corresponds to a feature. Sometimes it is assumed that the input space and the feature space are the same space, they are not differentiated, sometimes it is assumed that the input space and the feature space are different spaces, the instance is mapped from the input space to the feature space. The model is actually defined on the feature space.
This provides a good basis for the classification of machine learning algorithms, according to the specific conditions of the input space, the characteristic space and the output space, the specific condition of the algorithm is classified.

Classification of various learning-type algorithms

First of all, the following classification is my study of the relevant courses and their own learning process in the induction, not perfect, but can summarize a certain problem, I hope to summarize here, in case you can clear up ideas and improve.
The classification of the algorithm introduced is based on the general angle of the learning content of beginners, the classification angle from the common classification to the relatively unfamiliar classification method.

According to the different output space as the classification

• Second class classification (binary classification), commonly known as non-problem (say yes/no). Its output Space y={-1,+1}
• Multi-category Classification (Multiclass classification), Output space Y={1,2,...,k}
• Regression problem (regression), output space y=r, that is, the real range, the output is an infinite number of possible
• Structural Learning (structured learning), y=structures, the learning model can also be seen as a form of multi-class learning, which may involve a large number of categories

Example:

The two categories are widely used, such as judging whether spam, advertising investment can be profitable, learning system on the next question is correct. The second class classification is very important in machine learning and is the basis of other algorithms.
The application of multi-category classification, for example, according to a picture, the picture is Apple, orange or strawberry, and so on, like Google mailbox, Mail automatically into spam, important mail, social mail, promotional mail, etc. Multi-category classification is widely used in the identification of visual or auditory.
Regression analysis is widely used in predicting stock price and weather temperature.

Structural Learning (structured learning)

In natural language processing, automatic part-of-speech tagging is a typical example of structural learning. For example, given a machine a sentence, because the words in different sentences may have different parts of speech, so the method is used to understand the structure of the sentence characteristics. This learning method can be considered as a multi-category classification, but unlike a multi-category classification, the structure of the object may be large in size and its category is hidden behind the sentence.
Examples of this structural learning include, for example, the 3D stereoscopic structure of proteins in organisms and natural language processing.
Structural learning is described in some places as a callout (tagging) problem, where the input of the callout question is an observation sequence, and the output is a sequence of markers or sequences of states. The goal of labeling is to learn a model that allows the sequence of markers to be given as a predictor of the observed sequence.

Classify data labels according to their differences

The most common categories under this classification are supervised learning (supervised learning), unsupervised learning (unsupervised learning), and semi-supervised learning (semi-supervised learning). In the above discussed with the different output space as the basis for classification, the basic is to supervise the study, let us take a look at the other two.

Unsupervised learning (unsupervised learning)

cluster (clustering), {x[n]} = cluster (x), where the categories of data are not known, according to a certain rule to get a different classification.
It can be approximated as unsupervised multi-category classification.

density estimate (density estimation), {x[n]} = Density (x), where density (x) can be a probability density function or a probability function.
It can be approximated as an unsupervised bounded regression problem.

Exception Detection (outlier detection), {x[n]} = Unusual (x).
It can be seen as an approximate unsupervised class two classification problem.

Clustering is like a variety of articles on the network into different topics, commercial companies according to different customer information, the customer into different groups, and then adopt different promotional strategies.
A typical example of density estimation is the location-based traffic situation report, which predicts the area where the accident is more dangerous.
The exception detection example is, according to the network log situation, detects whether has the unusual intrusion behavior, this is an extreme "is not the problem", may use the unsupervised method to draw.

Semi-supervised learning (semi-supervised learning)

Semi-supervised learning (semi-supervised learning) is a learning method combining supervised learning with unsupervised learning. It mainly considers how to use a small number of labeling samples and a lot of unlabeled samples for training and classification problems. Semi-supervised learning is to improve the performance of machine learning algorithms by using unmarked large amounts of data.
There are five main algorithms for semi-supervised learning: probability-based algorithms, modified methods based on existing supervisory algorithms, methods for directly relying on clustering assumptions, methods based on multiple attempts, and graph-based methods.
Examples of semi-supervised learning, such as Facebook's recognition of face photos, may have only a small number of faces marked, most of them unmarked.

Enhanced learning (reinforce learning)

Reinforcement learning is a kind of machine learning method which takes environmental feedback as input, special and adaptive environment. The so-called reinforcement learning refers to the learning from the environment state to the behavior map, so that the system behavior from the environment to obtain the largest cumulative reward value. The method differs from the supervised learning technique by means of positive and inverse examples to tell what behavior to take, but by trial and error (Trial-and-error) method to discover the optimal behavior strategy.
The output here is not necessarily the output you really want, but rather a reward or punishment to tell the system whether it's good or not.
An online advertising system, for example, can be seen as a customer training the advertising system. This system gives the customer an advertisement, that is, the possible output, and the customer has no point or there is no money because of this advertisement, this assessment of the good or bad advertising delivery. This allows the advertising system to learn how to put more suitable ads.

Based on the different ways of communication with machines

Bulk Learning (batch learning), one-time batch input to the learning algorithm, can be called by the image of cramming learning.

Online Learning (online learning), in order, sequential learning, constantly to modify the model, to optimize.
hypothesis ' improves ' through receiving data instances sequentially
The first two classes of learning algorithms can be considered passive learning algorithms.

Active Learning (active learning), which can be viewed as a machine-to-question capability, specifying input x[n], asking for its output Y[N]
improve hypothesis with fewer labels (hopefully By asking questions strategically
This method is used when the acquisition cost of a label is very expensive

Many machine learning is the case of bulk learning.
Examples of online learning are like spam filters, mail is not all at once to train and identify, but in a sequence form, a lunar of the arrival, this way of progressive learning is constantly updated.
A simple example of active learning is that, like in the QQ space, there are photos of friends, that is, the machine to ask questions.

According to the different input space as the classification

• Specific characteristics (concrete features), each dimension of the input x is collated and analyzed by humans, and this analysis is often associated with a specialized field
• The original feature (Raw features), which requires the conversion of a person or machine, transforms the original feature into a specific feature, in which the identification of machine vision and sound signals belongs to the class
• Abstract features (features),

Examples of abstract features, such as the number of secondary students in an online teaching system, and the ID of the ad in the ad system. Using them all requires more action for feature extraction.

Additions to the original features

Here's how to add a bit of knowledge about the in-depth learning (deep learning) in raw features.
Deep learning is extracted automatically by the machine. It requires a lot of information or unsupervised learning to learn how to extract very specific features from it.
Deep Learning represents attribute categories or characteristics by combining lower-level features to form more abstract higher levels, to discover distributed feature representations of data.

Summary

The above is my beginner stage for the study of the classification of the type of algorithm, there may be inaccurate places, but also needs the reader to analyze their own judgment. Finally, I would like to say to myself, in the course of my study, I do not have to pursue the content of the record as detailed as possible, but in the process of recording written text can really deepen the understanding of the problem, and constantly self-thinking. After all, I write these things not for the sake of books, but to accumulate the key content of learning flexibly, and to do better knowledge management. Of course, it would be better if it helped the reader.

Reprint please indicate the author Jason Ding and its provenance
GitHub home page (http://jasonding1354.github.io/)
CSDN Blog (http://blog.csdn.net/jasonding1354)
Jane Book homepage (http://www.jianshu.com/users/2bd9b48f6ea8/latest_articles)

Classification of machine learning algorithms based on "machine Learning Basics"--on how to choose machine learning algorithms and applicable solutions