Classification, tagging, and Regression

Principles, practices, and practices of machine learning algorithms-classification, tagging, and regression 1. Classification Problems

Classification is a core issue of supervised learning. In supervised learning, when the output variable $ y $ gets a limited number of discrete values, the prediction problem becomes a classification problem.

Supervised Learning learns a classification decision-making function or classification model from data, called classifier ). Classifier predicts the output of new inputs. This process is called classification.

Classification includes two processes: Learning and classification. In the process of learning, a classifier is learned based on the known training sample dataset using an effective learning method. In classification, a learning classifier is used to classify new input instances.

For training dataset $ (x_1, y_1), (X_2, y_2), \ dots, (X_n, Y_n) $, the learning system learns a classifier $ P (Y | X) $ or $ Y = f (x) $; classifier learned by the classification system $ P (Y | x) $ or $ Y = f (x) $ classify the new input instance $ X _ {n + 1} $, that is, the predicted output class tag $ Y _ {n + 1} $.

The performance evaluation index of a classifier is generally the classification accuracy. It is defined as the ratio of the number of samples correctly classified by the classifier to the total number of samples for a given test dataset.

The common evaluation indicators for binary classification problems are precision and recall ). Generally, the following classes are positive and other classes are negative. The prediction or correctness or inaccuracy of the classifier on the test dataset is recorded as follows:

• TP -- predict the number of positive classes to positive classes;
• FN -- predict the number of positive classes as negative classes;
• FP: predict the number of positive classes;
• Tn-predict the number of negative classes;

Accuracy is defined:

$ P = \ frac {TP} {TP + FP} $

Recall rate is defined:

$ R =\frac {TP} {TP + fn} $

In addition, there is a $ F_1 $ value, which is the call average of the accuracy and recall rate, that is

$ \ Frac {2} {F_1 }=\ frac {1} {f} + \ frac {1} {r} $

$ F_1 =\frac {2tp} {2tp + FP + fn} $

When both precision and recall rates are high, $ F_1 $ is also high.

Many machine learning methods can be used to solve classification problems, including the $ K $ Nearest Neighbor Method, perception machine, Naive Bayes method, decision tree, logistic regression model, SVM, AdaBoost, Bayesian Network, and neural network.

For example, an example of text content classification. Text Classification divides a text into existing classes based on its features. Input is the feature vector of text, and output is the type of text. Generally, words in the text are defined as features, and each word corresponds to a feature. Word features can be binary: if a word appears in the text, the value is 1; otherwise, the value is 0; it can also be multi-value, indicating the frequency of the word appears in the text. Vividly, if there are many words like "stock", "bank", and "currency", this text may be economic. If the words like "tennis", "competition", and "athlete" frequently appear, this text may belong to sports.

2. annotation Problems

Tagging is also a problem of supervised learning. It can be considered that marking is a promotion of classification issues.

The input of the annotation problem is an observation sequence, and the output is a mark sequence or state sequence. That is to say, the output of classification is a value, while the output of annotation is a vector. Each value of a vector belongs to a tag type.

The tagging problem can also be divided into two steps: Learning and tagging. First, a training dataset is given.

$ T = (x_1, y_1), (X_2, y_2), \ dots, (X_n, Y_n) $

Here, $ X_ I = (x_ I ^ {(1)}, X_ I ^ {(2)}, \ dots, X_ I ^ {(n)}) ^ t, I = 1, 2, \ dots, N $ is the input observation sequence, $ y_ I = (y_ I ^ {(1)}, y_ I ^ {(2)}, \ dots, y_ I ^ {(n)}) ^ t $ is the corresponding output tag sequence. $ N $ is the sequence length and can have different values for different samples. The learning system constructs a model based on the training dataset, expressed as conditional probability distribution:

$ P (y ^ {(1)}, y ^ {(2)}, \ dots, y ^ {(n)} | x ^ {(1 )}, x ^ {(2)}, \ dots, x ^ {(n)}) $

Here, each $ x ^ {(I)} (I = 1, 2, \ dots, n) $ value is all possible observations, and each $ y ^ {(I )} (I = 1, 2, \ dots, n) $ indicates all possible tags, generally $ n \ ll N $. Based on the conditional probability distribution model obtained from learning, the tagging system finds the corresponding output tag sequence for the new input observation sequence. Specifically, for an observation sequence $ X _ {n + 1} = (X _ {n + 1} ^ {(1 )}, X _ {n + 1} ^ {(2)}, \ dots, X _ {n + 1} ^ {(n )}) ^ t $ locate the condition probability $ P (Y _ {n + 1} ^ {(1)}, Y _ {n + 1} ^ {(2 )}, \ dots, Y _ {n + 1} ^ {(n)} | X _ {n + 1} ^ {(1 )}, X _ {n + 1} ^ {(2)}, \ dots, X _ {n + 1} ^ {(n )}) $ the largest tag sequence $ Y _ {n + 1} = (Y _ {n + 1} ^ {(1 )}, Y _ {n + 1} ^ {(2)}, \ dots, Y _ {n + 1} ^ {(n)}) ^ t $.

The indicators of the evaluation annotation model are the same as those of the evaluation classification model. Common indicators include the accuracy, accuracy, and recall rate.

Common machine learning methods for tagging include hidden Markov models and Conditional Random Fields.

Part of speech tagging in natural language processing is a typical labeling problem: given a sentence composed of words, each word in the sentence is marked in parts of speech, that is, the part-of-speech mark sequence corresponding to a word sequence is predicted.

3. Regression Problems

Regression is also a type of supervised learning. Regression is used to predict the relationship between input and output variables, especially when the value of input variables changes, the value of output variables changes accordingly.

The regression model is a function that maps input variables to output variables. Regression learning is equivalent to function fitting: A function curve is used to fit known data and predict unknown data.

Regression Problems can be divided into one-dimensional regression and multiple regression based on the number of input variables. Linear Regression and Non-Linear Regression Based on the relationship between input variables and output variables.

The most common loss function in regression learning is the square loss. In this case, the regression problem can be solved by the famous least square method.

An example of using regression learning for stock prediction: assuming that you know the stock price (or average price for a period of time) of a company in the market at different time points in the past ), and information that may affect the company's shares between various time points (for example, the company's turnover in the previous week ). The goal is to learn a model from past data so that it can predict the stock price of the company at the next time point based on the current information. Specifically, the information that affects the stock price is considered as the independent variable (input feature), and the stock price is considered as the dependent variable (output value ). Using the past data as training data, you can learn a regression model and predict future shares. Actually, we know that it is difficult to make a satisfactory stock price prediction model. Because there are many factors that affect the stock, we may not be able to obtain useful information.

Classification, tagging, and Regression