Talk about unsupervised learning in machine learning

Machine learning is divided into supervised machine learning, unsupervised machine learning, and semi-supervised machine learning. The criterion for dividing it is whether the training sample contains human-labeled results.

(1) Supervised machine learning: a function is learned from a given set of training data, and when new data arrives, the result can be predicted based on this function. The training set requirements for supervised learning are both input and output, as well as features and goals. The goal of the training set is to be marked by people. Common supervised learning algorithms include regression analysis and statistical classification.  
(2) Unsupervised machine learning: The training set has no human-labeled results compared to supervised learning. The common unsupervised learning algorithms are clustering.  
(3) semi-supervised machine learning: A learning method combining supervised learning with unsupervised learning. It mainly considers how to use a small amount of sample labeling and a large number of unlabeled samples for training and classification problems. Semi-supervised learning is of great practical significance for reducing labeling cost and improving learning machine performance.  

> Today, let's talk about unsupervised machine learning:

The process of cluster analysis can be broadly divided into four steps, as shown in.
First, the data set needs to be preprocessed, usually including data dimensionality reduction, feature selection or extraction, and so on;
The second step is to design or select the clustering algorithm according to the characteristics of the dataset;
Third step, test and evaluation of clustering algorithm;
The fourth step, the presentation and interpretation of clustering results, Gain valuable knowledge from a data set through clustering analysis.

There are a lot of specific methods of clustering analysis, which is not introduced here.

Let's talk about the second major application direction of unsupervised machine learning: Autoencoder, Chinese is also an automatic encoder.

The first problem in machine learning is the problem of feature selection, and how to choose the characteristics of good expression ability of raw data becomes a headache in the process of machine learning practice. As 256*256 's grayscale map corresponds to a 65,000 input vector, in other words, we want to solve a problem on a 65,000-dimensional space. High-dimensional data not only need to collect more sample data, but also need more computing resources and storage resources. Obviously, an image does not need to be such a primitive feature to classify, we can do artificial feature selection to reduce the original features of the dimension. But this not only requires researchers to study the original data for a long time, many of the characteristics of the function is not easy to find, at the same time, the generalization of feature selection becomes very poor.

So, is there any way to extract the most information from the original message, the answer is yes, through a unsupervised machine learning method: Autoencoder, can solve our problems to a certain extent.

autoencoder is a unsupervised machine learning technique that uses a neural network to produce a low-dimensional representation of a high-dimensional input. The traditional dimension descent relies on linear methods, such as PCA, to find the direction of the largest variance in high-dimensional data. By choosing these orientations, PCA essentially depicts the direction in which the final information is contained. So we can find the number of a smaller dimension as the result of dimensionality reduction. However, the linearity of PCA method is also a great limitation on the type of feature dimension which can be extracted by itself. Autoencoder overcomes these limitations by introducing the inherent nonlinearity of neural networks.

autoencoder contains two main parts, encoder and decoder networks (coded networks and decoding networks). The encoder network is used during training and deployment, while the decoder network is used only during training. The function of the Encoder network is to discover a compressed representation of a given data.

Its basic structure:

Autoencoder structure diagram

We will input a encoder encoder, we will get a code, this code is a representation of the input, then how do we know that this code means input? We add a decoder decoder, this time decoder will output a message, then if the output of this information and the input signal is very similar to the beginning (ideally the same), it is obvious that we have reason to believe that this code is reliable. So, by adjusting the parameters of the encoder and decoder to minimize the refactoring error, we get the first representation of the input signal, which is code code. Because there is no label data, the source of the error is directly reconstructed and compared to the original input.

stackedautoencoders: Cascading of multiple autoencoder means

stacked Autoencoders has more powerful expression ability and all the advantages of deep network, the first layer can be learned to one-order features, the second layer can learn the two-second characteristics, etc., for the image, the first layer may learn to edge, the second layer may learn how to combine edges to form outlines, points, Higher-level students may learn more image and more meaningful features, learn the features to facilitate better processing of images, such as image classification, retrieval and so on.

If you add a regular item to the output of the Autoencoder. A sparse Autoencoder sparse automatic encoder is obtained, which is a very good dimensionality reduction method in image processing and NLP field. such as SVM processing text classification, using TF_IDF to encode the original text, here TF_IDF can be regarded as an artificial encoder, can achieve good results.

So, what are the advantages of such automatic coding methods such as Autoencoder and the traditional linear dimensionality reduction method?

Take Autoencoder's 2-D representation and PCA's 2-D representation as an example of the performance of a well-known image dataset Mnist a handwritten digital data set:

Autoencoder Right, PCA left

autoencoder is a new direction of unsupervised machine learning and provides a very good method of data modeling for the application of machine learning in complex problems.

On tablets, all from other blogs or papers, in this one by one thanked.

Talk about unsupervised learning in machine learning