Different classification problems: Multi-class classification, multi-label classification, multi-sample learning, multi-task learning

Multi-Class classification (Multiclass classification)

A sample belongs to and belongs to only one of several classes, and one can belong to only one class, and the different classes are mutually exclusive.

Typical method: One-vs-all or One-vs.-rest:

Divide a number of questions into N two class classification problem, train n two class classifier, for the class I, all the samples belonging to Class I are positive (positive) samples, the other samples are negative (negative) samples, each Class II classifier separates the samples belonging to Class I from other classes.

One-vs-one or All-vs-all:

The N (N-1) class Two classifier is trained, and each classifier distinguishes between a pair of classes (I,J).

Multi-label Classification (Multilabel classification)

Also known as multi-label learning, multi-tagging learning, different from multi-class classification, a sample can belong to more than one category (or label), different classes are related.

A method of typical method problem transformation

The core of the problem transformation method is "transforming the sample data to adapt to the existing learning algorithm". The idea of this kind of method is to adapt the multi-marker training sample to the existing learning algorithm, which is to solve the problem of multi-tagging learning.

The representative learning algorithm has a first order method binary relevance, which transforms the multi-tagging learning problem into "two-class classification (binary classification)" problem solving; the second-order method calibrated Label Ranking, This method transforms the multi-tagging learning problem into "tag sorting (labelranking) problem solving, and the higher order method random K-labelset, which transforms the multi-tagging learning problem into a" multi-class classification (Multiclass classification) "Problem solving.

Algorithm Adaptation method

The core of algorithm adaptation method is "transforming existing single-marker learning algorithm to adapt to multi-tagged data". The basic idea of this kind of method is to improve the traditional machine learning method so that it can solve the multi-tagging problem.

The representative learning algorithm has a first-order method ML-KNN}, which will transform the lazy learning (lazy learning) algorithm K nearest neighbor to adapt to multi-tagged data, and the second-order method RANK-SVM, the method will "nuclear learning (Kernel learning)" The algorithm SVM is modified to adapt to the multi-tagged data, and the advanced method leads, which transform Bayesian learning (Bayes Learning) Algorithm Bayes network to adapt to multi-tagged data.

Multi-sample learning (Multi-instance learning)

In this type of learning, the training set consists of a number of concept-tagged packages (bag), each containing several examples without a concept tag. If there is at least one positive example in a package, the package is marked as positive (positive), and if all the examples in a package are reversed, the package is marked as reversed (negative). By learning the training package, we hope that the learning system can predict the concept mark of the package outside the training set as correctly as possible.

Multi-tasking Learning (multi-task learning)

Multitasking Learning (multi-task learning) is a machine learning approach that is relative to single-tasking learning (Single-task learning). In the field of machine learning, the standard algorithm theory is to learn one task at a time, that is, the output of the system is a real number case. The complex learning problems are decomposed into theoretical independent sub-problems, then each sub-problem is studied, and finally the mathematical model of complex problem is established by the combination of the learning result of the pair problem. Multi-task learning is a joint learning, multiple tasks parallel learning, the results of mutual influence.

In a simple contrast to the school data that we often use, school data is a dataset that predicts the regression problem of student performance, with a total of 139 secondary students, each of which can be seen as a predictive task. Single-task learning is about ignoring the possible relationships between tasks. Learn the predictions of 139 regression functions for fractions, or simply put all the data from 139 schools together to learn a regression function to predict. While multi-task learning emphasizes the connection between tasks, through joint learning, at the same time, 139 tasks to learn different regression functions, both to consider the differences between tasks, but also considering the relationship between tasks, which is the most important idea of multi-tasking learning.

The early research work of multi-task learning stems from an important problem in machine learning, that is, the study of "inductive bias (inductive bias)". The process of machine learning can be regarded as the analysis of the empirical data related to the problem, and the process of the model which reflects the essence of the problem is summed up. The function of inductive bias is to instruct the learning algorithm how to search in the model space, and the performance of the search model will be directly affected by inductive bias, and any learning system lacking inductive bias can not be effectively studied. Different learning algorithms (such as decision trees, neural networks, support vector machines, etc.) have different inductive biases, and people need to determine which learning algorithm to use when solving practical problems, in fact, they choose different inductive bias strategies subjectively. A very intuitive idea is whether the process of determining inductive biases can be done automatically through the learning process, i.e. the idea of learning how to learn (learning to learn). Multi-task learning provides a feasible way for the realization of the above ideas, that is, to use the useful information contained in the relevant tasks to provide a stronger inductive bias for the learning of the tasks concerned.

Typical methods

At present, the multi-task learning method can be summarized into two categories, one is to share the same parameters among different tasks (common parameter), and the other is to excavate the hidden common data features (latent feature) between different tasks.

Different classification problems: Multi-class classification, multi-label classification, multi-sample learning, multi-task learning