Recommended system evaluation Indicators-accuracy (Precision), Recall (Recall), F value (f-measure) _ Machine learning

1. Accuracy rate and Recall rate (Precision & Recall)

Accuracy and recall rates are two measures widely used in the field of information retrieval and statistical classification to evaluate the quality of the results. The accuracy is to retrieve the number of related documents and the total number of documents retrieved, to measure the precision of the retrieval system; Recall is the ratio of the number of documents retrieved and the number of documents in the document library, and the recall of the retrieval system is measured.

In general, precision is the number of retrieved items (such as documents, Web pages, etc.) is accurate, recall is the number of all accurate entries have been retrieved.

The correct rate, recall rate and F value are the important evaluation indexes to select the target in the mixed environment. Take a look at the definition of these indicators first:

1. Correct rate = The correct number of messages extracted/The number of  messages extracted     

2. Recall = number of information in the correct Information Bar/sample extracted      

The value is between 0 and 1, the closer the value is to 1, the higher the precision or recall.

3. F  = correct rate * Recall rate * 2/(correct rate + recall rate) (f value is the correct rate and recall rate of the harmonic mean)

Take this example: there are 1400 carp, 300 shrimp and 300 turtles in a pond. It is now for the purpose of catching carp. 700 Carp, 200 shrimp and 100 turtles were caught in a large net. Then, the indicators are as follows:

Correct rate = 700/(700 + 200 + 100) = 70%

Recall rate = 700/1400 = 50%

F value = 70% * 50% * 2/(70% + 50%) = 58.3%

Let's see if all the carp, shrimp and turtle in the pool are catch, what's the difference:

Correct rate = 1400/(1400 + 300 + 300) = 70%

Recall rate = 1400/1400 = 100%

F value = 70% * 100% * 2/(70% + 100%) = 82.35%

Thus, the correct rate is the proportion of target results in the results of the capture; recall, as the name suggests, is the proportion of the target category from the area of concern, and the F value is the evaluation index of the combination of the two indicators, which is used to comprehensively reflect the overall index.

Of course, we hope that the higher the precision the better, and the higher the recall the better, but in fact the two are contradictory in some cases. For example, in extreme cases, we only search for a result, and it is accurate, then precision is 100%, but recall is very low, and if we return all the results, such as recall is 100%, but precision will be very low. Therefore, in different occasions need to judge their own hope precision higher or higher recall. If you are doing experimental research, you can draw Precision-recall curves to help with the analysis.

2. Comprehensive Evaluation Index (F-MEASURE)

P and R indicators sometimes have contradictory situations, so that they need to be considered synthetically, the most common method is F-measure (also known as F-score).

F-measure is the weighted harmonic averaging of precision and recall:

When the parameter α=1, is the most common F1, also namely

It is F1 that the results of P and R are synthesized and the test method is more effective when the F1 is higher.

3, E value

The E value represents the weighted average of the precision P and recall R, when one of them is 0 o'clock and the e value is 1, and its formula is calculated:

The larger the B, the greater the weight of the precision.

4. Average correct rate (Average Precision, AP)

The

Average correct rate represents the average of the correct rate on points with different recall ratios.