Classification accuracy evaluation and multi-class classification performance evaluation of remote sensing

We know that the classification of remote sensing is uncertain, and its uncertainty is closely related to the evaluation method chosen. This paper mainly introduces four aspects, firstly introduces the development of the classification accuracy evaluation of remote sensing, and then introduces the evaluation method of remote sensing classification accuracy and the indexes of multi-classification performance evaluation respectively, and finally summarizes the similarities between the two points.

First, we understand the development process of classification accuracy evaluation of remote sensing image.
The classification accuracy evaluation Method of remote sensing image interpretation is divided into four stages in the study of quantitative remote sensing model and application uncertainty in Liuchin etc.:
The first stage is mainly visual, for example, in the remote sensing image processing software, the classification results can be qualitatively evaluated by the rolling shutter mode, which has strong subjectivity.
The second stage is the qualitative to quantitative transition, which is to evaluate the classification accuracy by comparing the area proportion of each land category in the thematic map and the proportion of the actual land category. However, the method has some limitations, which may obscure the true accuracy of the classification results, for example, the area proportion of the figure classification is more consistent with the actual, but it is divided in the wrong position.
The third stage, on the basis of the second stage, compares the categories of the ground classes with the actual classes, and derives various precision measurements.
The forth stage is derived from the base of the third stage, with the error matrix as the core, and the various measurements are developed using the information of the error matrix.
then, we mainly summarize the accuracy evaluation of remote sensing classification based on the error matrix.
The error matrix, also known as the confusion matrix, has the following pattern:

then, we briefly introduce the performance evaluation criteria for multi-class classification.
In the performance evaluation of multi-class classification, we often use precision (accuracy), that is, the total number of samples/categories with correct classification. But we know that judging the accuracy of the classification in this way will lose the classification accuracy of the categories with less sample size, and in actual production life, this error is very deadly. For example, we have 100 patients to determine whether they have malignant tumors (the actual number of patients with malignant tumors is 4), even if we determine all patients without malignant tumors, our classification accuracy has reached 96%, looking at high classification accuracy, But obviously we know that the result of this classification is not satisfied with our production life requirements.
Therefore, we introduce the macro average (macro-average) and the micro-average (micro-average).
Generally in the calculation of the macro average is the F1 value, the macro average formula is:

For TP, FN, TN, FP, we know that in the multi-classification problem, the classification results generally have 4 kinds of situations:
Samples belonging to Class C are correctly classified into Class C, and this class of samples is TP;
Samples that do not belong to Class C are incorrectly categorized into Class C, and the number of samples in this category is FN;
Samples belonging to category C are incorrectly categorized into other classes of Class C, and the sample count is TN;
Samples that do not belong to category C are correctly categorized into the other classes of category C, and the number of samples in this class is FP.
The micro-average calculation formula is micro-average= (TP + FP)/(TP + TN + fp + FN), can be seen as the micro-average equivalent to the direct calculation accuracy.
Finally, we can see that the accuracy rate in multi-classification is the producer precision in remote sensing classification, and the recall rate in multi-classification is the user's accuracy in remote sensing classification.