Evaluation method of results in deep learning image processing (classification or detection)-map introduction

There is more than one label for a picture in the

Multi-label Image classification (Multi-label image classification) task, so the evaluation cannot be categorized by the standard single-label image, which is mean accuracy, which uses a similar approach to information retrieval- MAP (mean Average Precision). Although the literal meaning of map and mean accuracy look similar, but the calculation method is much more cumbersome. p-r Curve Drawing uses a trained model to get the confidence score of all test samples, and in this case there are 20 test samples. (Each class of p-r curves, APS are calculated separately)
for the class confidence score sort, get:
Calculate top-1 to Top-n (N is the number of all test samples, this article is 20) corresponding precision and recall, The two criteria are defined as follows:

The intuitive understanding is that the first time we sort the first sample of the confidence as a threshold to divide the positive and negative samples, at this time, only the first judgment is positive, The others are negative samples (because the confidence of the other samples are less than the confidence of the first sample), the recall (1/1=1) and precision (1/1=1) are computed for this threshold And then the second time the second sample confidence as the threshold for dividing the positive and negative samples, and so on to the last one. Obviously with the decrease of the threshold, the more we select the sample, the more recall will be higher, and the precision will show a downward trend overall. The recall as the horizontal axis, precision as the ordinate, you can get the usual precision-recall curve. The P-r curve for this example is as follows: Calculation of the
AP

The PASCAL VOC Challenge has been replaced by new calculations since 2010. The new calculation method assumes that the N samples have m positive examples, then we will get M recall values (1/m, 2/m, ..., m/m), for each recall value R, we can calculate the corresponding (R > R) of the maximum precision, The last AP value is then averaged over the M precision value. The calculation method is as follows:

The portion of the corresponding Precision-recall curve that is used to calculate the AP is as follows (each recall node is connected with the maximum value of precision):
Calculation of Map

The APS measure the quality of the models that are learned in each category, and map measures the quality of the model in all categories, and the calculation of the map after the AP becomes very simple, which is to take the average of all APs.

Note: This article changes from the original text, there are additions and deletions.