Machine Learning Algorithm Summary: Artificial neural network, deep learning and other

Learning Style

Depending on the type of data, there are different ways to model a problem. In the field of machine learning or artificial intelligence, people first consider the way of learning algorithms. In the field of machine learning, there are several main ways of learning. It is a good idea to classify the algorithm according to the learning style, so that people can choose the most suitable algorithm according to the input data to get the best results when modeling and algorithm selection.

Supervised Learning:

Under supervised learning, input data is called "training data", each set of training data has a clear identification or result, such as "spam" in the anti-spam system, "non-spam", to handwritten digit recognition "1", "2", "3", "4" and so on. When establishing a predictive model, supervised learning establishes a learning process, compares the predicted results with the actual results of the "training data", and adjusts the forecast model continuously until the predicted result of the model reaches an expected accuracy rate. Common application scenarios for supervised learning such as classification problems and regression issues. Common algorithms include logistic regression (logistic regression) and reverse transmission neural network (back propagation neural receptacle)

Non-supervised learning:

In unsupervised learning, data is not specifically labeled and the learning model is designed to infer some of the intrinsic structure of the data. Common application scenarios include the Learning of association rules and Clustering. Common algorithms include Apriori algorithm and K algorithm.

Semi-supervised learning:

In this learning mode, the input data is identified and part is not identified, the learning model can be used for prediction, but the model first needs to learn the internal structure of the data in order to reasonably organize the data to predict. The application scenarios include classification and regression, and the algorithm includes some extensions to the commonly used supervised learning algorithms, which first attempt to model the unsigned data and then predict the identified data. such as graph theory inference algorithm (graph inference) or Laplace support vector machine (Laplacian SVM.) And so on.

Intensive Learning:

In this learning mode, input data as feedback to the model, unlike the monitoring model, the input data is only as a check model of the wrong way, under the reinforcement learning, input data directly feedback to the model, the model must be adjusted immediately. Common application scenarios include dynamic systems and robot control. Common algorithms include q-learning and time lag learning (temporal difference learning)

In the case of enterprise Data application, the most common one is probably supervised learning and unsupervised learning model. In the field of image recognition, semi-supervised learning is a hot topic due to the existence of a large number of unidentified data and a small number of identifiable data. Reinforcement learning is more used in robotic control and other areas where system control is needed.

Algorithm similarity

According to the function of the algorithm and the similarity of form, we can classify the algorithm, such as tree based algorithm, neural network algorithm and so on. Of course, the scope of machine learning is very large, some algorithms can not be clearly classified into a certain category. For some classifications, the same classification algorithm can be used for different types of problems. Here, we try to classify commonly used algorithms in the easiest way to understand them.

Regression algorithm

The regression algorithm is a kind of algorithm that tries to use the measurement of the error to explore the relationship between variables. The regression algorithm is a tool for statistical machine learning. In the field of machine learning, people say that regression, sometimes refers to a class of problems, sometimes refers to a class of algorithms, which often makes beginners confused. Common regression algorithms include: least-squares (ordinary least square), logistic regression (logistic regression), stepwise regression (stepwise regression), multivariate adaptive regression spline (multivariate Re-use regression splines) and local scatter smoothing estimates (locally estimated scatterplot)

case-based algorithm

Case-based algorithms are often used to model decision problems, such models often select a batch of sample data and then compare the new data with the sample data according to some approximations. Find the best match in this way. Therefore, case-based algorithms are often referred to as "winner-take-all" learning or "learning based on memory". Common algorithms include k-nearest neighbor (KNN), Learning vector quantization (Learning vector quantization, LVQ), and self-organizing mapping algorithms (self-organizing map, SOM)

Regularization method

The regularization method is an extension of other algorithms (usually the regression algorithm), which adjusts the algorithm according to the complexity of the algorithm. The regularization method usually rewards the simple model and punishes the complex algorithm. Common algorithms include: Ridge regression, least differs shrinkage and Selection Operator (LASSO), and resilient networks (elastic Net).

Decision Tree Learning

Decision Tree algorithm uses tree structure to establish decision model based on the attribute of data, and decision tree model is often used to solve the problem of classification and regression. Common algorithms include: Classification and regression trees (classification and regression tree, CART), ID3 (iterative Dichotomiser 3), C4.5, chi-squared Automatic Consortium Detection (Chaid), Decision Stump, Random Forest (Random Dara), multivariate adaptive regression spline (MARS), and gradient propulsion (gradient Boosting Machine, GBM)

Bayesian method

Bayesian algorithm is a kind of algorithm based on Bayesian theorem, which is mainly used to solve the problem of classification and regression. Common algorithms include: naive Bayesian algorithm, average single dependency estimation (averaged one-dependence estimators, Aode), and Bayesian belief (receptacle).

Kernel based algorithm

The most notable of the kernel based algorithms is the support vector machine (SVM). The kernel based algorithm maps the input data to a higher order vector space, in which some classification or regression problems can be solved more easily. Common kernel-based algorithms include: Support vector machines (Support vector Machine, SVM), Radial basis functions (radial based function, RBF), and linear discriminant analysis (Linear discriminate analyses , LDA) and so on.

Clustering algorithm

Clustering, like regression, is sometimes described as a kind of problem, and sometimes it is a class of algorithms. The clustering algorithm usually merges the input data according to the central point or the layered method. So the clustering algorithm attempts to find the intrinsic structure of the data in order to classify the data according to the most common denominator. The common clustering algorithm includes K algorithm and expectation maximization algorithm (expectation maximization, EM).

Association Rules Learning

Association rule Learning finds useful association rules in a large number of multivariate datasets by looking for rules that best explain the relationship between data variables. Common algorithms include Apriori algorithm and Eclat algorithm.

Artificial neural network

Artificial neural network algorithm simulates biological neural network, and is a kind of pattern matching algorithm. Often used to solve classification and regression problems. Artificial neural network is a large branch of machine learning, there are hundreds of different algorithms. (Depth learning is one of the algorithms, we will discuss it alone), important artificial neural network algorithms include: Perceptron Neural network (perceptron neural receptacle), reverse transmission (back propagation), Hopfield network, Self-organizing Mappings (self-organizing map, SOM). Learning vector quantization (Learning vector quantization, LVQ)

Deep learning

The depth learning algorithm is the development of artificial neural network. In recent years has won a lot of attention, especially Baidu began to exert deep learning, but also in the domestic cause a lot of attention. In today's increasingly inexpensive computing power, deep learning attempts to build a much larger and more complex neural network. A lot of depth learning algorithms are semi supervised learning algorithms, which are used to deal with large datasets with little or no identifying data. Common depth learning algorithms include: limited Boltzmann (restricted Boltzmann Machine, RBN), Deep belief NX (DBN), convolution network (convolutional receptacle), Stack type Automatic encoder (stacked auto-encoders).

Reduced dimension algorithm

Like clustering algorithms, the dimensionality reduction algorithm attempts to analyze the intrinsic structure of the data, but the reduced dimension algorithm attempts to use less information to induce or interpret data in unsupervised learning. Such algorithms can be used for visualization of high-dimensional data or for simplifying data for supervised learning purposes. Common algorithms include: PCA (principle Component Analysis, PCA), Partial least squares regression (Partial least Square Regression,pls), Sammon Mapping, multidimensional scaling ( Multi-dimensional scaling, MDS), projection tracking (projection Pursuit), etc.

Integration algorithm

The integration algorithm trains the same sample independently with some relatively weak learning models, and then integrates the results to predict the whole. The main difficulty of integration algorithm is how to integrate the weaker learning model and how to integrate the learning results. This is a very powerful algorithm, but also very popular. Common algorithms include: Boosting, bootstrapped Aggregation (bagging), AdaBoost, stacked generalization (stacked generalization, blending), gradient propulsion (gradient Boosting Machine, GBM), Random Forest (Random Dara).