Accuracy, recall and F-values

Original: http://peghoty.blog.163.com/blog/static/49346409201302595935709/

The correct rate, recall rate and F value are the important evaluation indexes of the target in the mixed environment. Consider the definition of these indicators first:

1. Correct rate = number of individuals correctly identified/number of individuals identified

2. Recall rate = Number of individuals correctly identified/number of individuals present in the test set

3. F value = correct rate * Recall rate * 2/(correct rate + recall rate) (f value is the harmonic mean of correct rate and recall rate)

Example: There are 1400 carp, 300 shrimp and 300 turtles in a pond. Now for the purpose of catching carp. Seaeagl e sprinkled a large net, caught 700 carp, 200 shrimp, 100 turtles. So, these indicators are as follows:

Correct rate = 700/(700 + 200 + 100) = 70%

Recall rate = 700/1400 = 50%

F value = 70% * 50% * 2/(70% + 50%) = 58.3%

Let's see if seaeagle clean sweep all the carp, shrimp and turtles in the pond, and how these indicators change:

Correct rate = 1400/(1400 + 300 + 300) = 70%

Recall rate = 1400/1400 = 100%

F value = 70% * 100% * 2/(70% + 100%) = 82.35%

It can be seen that the correct rate is the proportion of the target achievement in the results of the capture; recall, as the name implies, is the proportion of the target category recalled from the area of concern, while the F-value is the evaluation index of the two indicators, which is used to comprehensively reflect the overall indicators.

Add:The following is the Baidu encyclopedia in the recall rate of the detailed http://baike.baidu.com/view/29947.htm recall rateBasic Concepts

Recall rate

Recall (Recall) and precision (precise) are the two metrics widely used in the field of information retrieval and statistical classification to evaluate the quality of the results. The recall rate is the ratio of the number of related documents retrieved and the number of related documents in the document library, which is measured by the recall of the retrieval system. The accuracy is the ratio of the number of related documents retrieved and the total number of documents retrieved, and the precision of the retrieval system is measured. calculation Methodrecall rate and accuracy

Recall

assume: When retrieving a document from a large data collection, you can divide the document into four groups:-related documents retrieved by the system (A)-unrelated documents retrieved by the system (B)-related but no document retrieved by the system (C)-documents that are not relevant and are not retrieved by the system (D)Then:-Recall rate R: with the number of related documents retrieved as the numerator, the total number of all related documents as the denominator, i.e. R = A/(A + C)-accuracy P: The total number of retrieved documents is used as the denominator with the number of related documents retrieved as the numerator. That is, p = A/(A + B).For example:a database has 500 documents with 50 documents that meet the defined issues. The system retrieves 75 documents, but there are only 45 issues that meet the definition. Recall rate r=45/50=90%accuracy p=45/75=60%in this case, the system retrieval is more effective and the recall rate is 90%. But the result is a lot of noise, and nearly half of the search results are irrelevant. Studies have shown that it is difficult to obtain a high recall rate without sacrificing accuracy. recall rate of the search systemfor a retrieval system, recall and accuracy are not the same: when the recall rate is high, the accuracy is low, the accuracy is high, the recall rate is low. Therefore, the accuracy of a retrieval system is measured by the average of 11 accuracy of 11 recall rates. We can also fuse these two measures into one measure, such as the F measure (f-measure). for the search engine system, because no search engine system can collect all the Web pages, so the recall rate is difficult to calculate. The current search engine system is very concerned about precision . there are many factors affecting the performance of a search engine system, the most important is the information retrieval model, including the presentation method of document and query, the matching strategy of evaluating document and user query relevance, the sorting method of query results and the mechanism of user's correlation feedback. Although there is no inevitable relationship between "recall rate" and "accuracy rate", the two indexes in large-scale data collection are constrained by one another . because the "retrieval strategy" is not perfect, and when more relevant documents are to be retrieved, the "search strategy" is usually relaxed, and some unrelated results are often associated with it, thus making the accuracy affected. when you want to get rid of irrelevant documents in your search results, it's important to make the "search strategy" stricter, so that some related documents can no longer be retrieved, and the recall is affected. The retrieval and selection of large-scale data collection involve the two indexes of "recall rate" and "accuracy rate". And because of the two indicators of mutual constraints, we usually also according to the need for "search strategy" to choose a suitable degree, not too strict and not too loose, to seek a balance between the recall rate and the accuracy of the middle。 This equilibrium point is determined by the specific needs.

Accuracy, recall and F-values