Pattern Recognition Evaluation Method ROC curve, Det curve, FPPW, fppi etc._ image processing algorithm

Pattern Recognition Evaluation Method ===> ROC curve DET curve FPPW Fppi

The final performance evaluation of Pattern recognition algorithm is the key because of the work done by the individual in pattern recognition. But the internet is difficult to find specific, detailed evaluation process, methods and code, so I intend to prepare the title as shown in the evaluation method of finishing work, then will be introduced in the method, the basic code (MATLAB version) to help more like me to these methods some confused people.

Provide the Roc,det curve drawing function in MATLAB for the time being: Perfcurve ()

Specific use method, Chinese link: http://ilovematlab.cn/thread-76781-1-1.html

Mathwork Manual (the document originates from matlab2013.) The previous version should have this function, but I did not find in my 2008a version, 2010 version should have): http://www.mathworks.cn/cn/help/stats/perfcurve.html

Self-compiling ROC curve and related intermediate result calculation code.

Main function:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%function:roc curve related
%   date:2013.07.01
% author:xin Yang, School of Medicine, Szu
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% of ROC, Label8score curve drawn according to Det file
Get data file path
[filename, pathname] = Uigetfile ({' *.txt ', ' Model Files (*.txt) '; ')
*.* ', ' All Files (*.*) '}, ' select ROC data source ', ' F:\Result\FPPW\ ');
If IsEqual (filename,0) | | IsEqual (pathname,0)
    return;% If you click "Cancel"
else
    l8s_file = [pathname, filename];
End
dot_num = m;
Yx_roc (L8s_file, dot_num);
% Yx_det_curve (l8s_file, dot_num);

Yx_roc function:

% Markup & Predictive scoring file drawing Roc curve function Yx_roc (label8scorefile, dot_num)% load Mark & forecast score Data file 1: Positive 0: Negative mat_l8s = Load (Label8
Scorefile);
Sample_num = Size (mat_l8s, 1);

% draw positive and negative marker distribution trend chart, used to observe the cross degree of positive and negative samples plot_pn_pdf (mat_l8s);
Label = mat_l8s (:, 1);% artificial Mark Score = mat_l8s (:, 2);% algorithm score% upper and lower Upper = max (Score);
Lower = min (Score);
% widen the upper and lower limit, easy to reach the limit Upper = Upper + 0.1;
Lower = Lower-0.1;
% points bins = dot_num;
% Stepping step = (upper-lower)/bins;
% threshold change T = Lower:step:Upper;
T_num = bins + 1;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% TPR, FPR TPR = Zeros (T_num, 1);
FPR = Zeros (t_num, 1);
    % traversal threshold for i = 1:t_num t_temp = T (i); TP = 0; FN = 0;% 0 FP = 0;
    TN = 0; % traversal Sample for j = 1:sample_num if (Label (j) = = 1) if (Score (j) >= t_temp) TP = TP + 1;% Zhenyang Else fn = fn + 1;% false Yin End ElseIf (Label (j) = = 0) if ( Score (j) >= t_temp) fp = FP + 1;% false Positive else tn = tn + 1;% True yin End End end% * * TPR (i) = T
    p/(TP + FN);
FPR (i) = fp/(FP + TN);
End% ROC curve Figure plot (FPR, TPR, ' C ', ' LineWidth ', 2);
Axis ([0, 1, 0, 1]);
Xlabel (' 1-specificity ') ylabel (' sensitivity ') title (' ROC ') grid on% Youden index figure X = TPR-FPR;
Plot (X, ' R ', ' LineWidth ', 2);
Title (' Youden index ');
% TPR, FPR with threshold changes, easy to observe, determine thresholds figure, PLOT3 (FPR, TPR, T, ' Color ', ' r ', ' LineWidth ', 2); Xlabel (' \FONTSIZE{14}\BFX axis--FPR ') ylabel (' \fontsize{14}\bfy axis--TPR ') Zlabel (' \fontsize{14}\bfz axis--Threshold ') title
 (' \FONTSIZE{14}\BF threshold selection ')

Plot_pn_pdf function:

% The probability density function curve of positive and negative marks based on Label2score
plot_pn_pdf (mat_label2score)

[La_p_r, La_p_c] = Find (mat_ Label2score (:, 1) = = 1);
[La_n_r, La_n_c] = Find (Mat_label2score (:, 1) = = 0);

P_score = Mat_label2score (La_p_r, 2);
N_score = Mat_label2score (La_n_r, 2);

[P_count, P_bin] = hist (P_score, m);
[N_count, N_bin] = hist (N_score, m);
P_count = (P_count-min (p_count))/(Max (P_count)-min (P_count));
N_count = (N_count-min (n_count))/(Max (N_count)-min (N_count));

Figure 
values = SPCRV ([[P_bin (1) p_bin p_bin (end)];[ P_count (1) p_count p_count (end)]],3);
Plot (values (1,:), values (2,:), ' Color ', ' r ', ' LineWidth ', 2);
Hold on 
values = SPCRV ([[N_bin (1) n_bin n_bin (end)];[ N_count (1) n_count n_count (end)]],3);
Plot (values (1,:), values (2,:), ' Color ', ' B ', ' linewidth ', 2);
Legend (' \fontsize{14}\it  p-labeled ', ' \fontsize{14}\it  n-labeled ')
xlabel (' \fontsize{14}\ Bfscore ')
ylabel (' \fontsize{14}\bffrequency ')
title (' \fontsize{14}\bf positive, negative marker distribution curve ')

Intermediate Result:

(1) (2)

(3) (4)

(1) The graph depicts the predicted score distribution (probability density function) of positive and negative test data, and indicates the intersection degree of positive and negative test sample score. The degree of intersection is determined by the integral of the cross part of the two curves. The more the crossover, the worse the classifier effect, the better the effect.

(2) The picture is ROC Curve, the closer to the upper left corner, the better the classifier performance.

(3) The graph is the ROC curve application, chooses the actual application threshold value one of the reference methods: Youden index. The threshold corresponding to the highest point of the curve is the best applied threshold. Youden Index can refer to the following article 6th page: http://www.medicalbiostatistics.com/roccurve.pdf

(4) The graph is based on ROC, selects the curve of threshold value. X, Y is false positive rate, Zhenyang rate, Z axis is the corresponding threshold value. It is always remembered that the ROC curve is produced because the ROC curve is drawn from a number of sets of Zhenyang and false positive rate points, which are derived from the threshold values as variables. So any point on the ROC curve has a corresponding threshold. To draw the ROC curve, in addition to seeing the classifier performance intuitively, another important role is, according to the ROC curve, select a reasonable threshold for practical detection, application, determination.

An additional two links to posts that are useful to the ROC and Precision-recall:

[1] http://www.zhizhihu.com/html/y2012/4076.html

[2] http://blog.csdn.net/abcjennifer/article/details/7834256

=============== 28-aug-2015 Increased ==================//

The other commonly used evaluation criteria for target detection are FPPW and Fppi, which can be referred to in detail: pedestrian DETECTION:A Benchmark
Both focus on the frequency with which FP (False Positive) occurs.

FPPW (False Positive per Window)

Basic meaning: Given a certain number of negative sample images of N, the classifier will determine the negative sample as the number of "positive" FP, the ratio of fp/n is FPPW. Meaning is the same as the false positive rate in ROC. In Fppw, a picture is a sample.

Fppi (False Positive per Image)

Basic meaning: Given a certain number of sample sets of N, containing n images, each image contains or does not contain detection targets.

Each image needs to be calibrated:

1. Contains the number of targets;

2. The exact location of the target L.

Then run the classifier on each image, detect the target and get position p. Then, check whether the test results in each image "hit" The calibrated target:

A. If there is no goal in the image, and the classifier gives n "target" detection results, then false Positive times +n;

B. If there is a target within the image, then determine whether P hit L, judging criteria refer to the above article (mainly see p and L overlap rate). If the judgment is not hit, false Positive number +1.

Last Fppi = (False Positive number)/N.

Compared with FPPW, Fppi is closer to the actual application of classifier.