Summary of image similarity measurement and template matching

Summary

This paper mainly summarizes the image similarity measurement and template matching method used in target tracking and detection, and gives the specific code implementation based on OPENCV.

Introduction

Template matching is a technique that looks for the most similar image patches in the source image, and is often used to identify, track and detect targets. The most similar must be based on some similarity criterion, that is, the need for similarity measurement. In addition, the search needs to be on the image progressive, column-by-row patch window scanning, of course, do not necessarily need to row-by-column scan, when the error of a few pixels than the speed of the calculation is not important to set the scan line and step values to speed up the scanning and calculation of time consumption. The following is a description of similarity measurement and template matching (all images are assumed to be grayscale).

BodyMeasurement of image similarityHistogram Method

Method Description: There are two image patches (but also the whole image), the histogram of two images are computed separately, and the histogram is normalized, and then the similarity measurement is carried out according to the criterion of some distance measure.

Method of thinking: Based on a simple vector similarity to the image similarity measurement.

Advantages: Histogram can be well normalized, such as 256 bin bar, so even the different resolution of the image can be directly through its histogram to calculate the similarity, the calculation is moderate. More suitable for describing images that are difficult to automatically segment.

Disadvantage: The histogram reaction is the image gray-scale worthy of probability distribution, and there is no image of the spatial location of the information in the inside, and therefore, often a miscarriage of error; From the information theory, through the histogram conversion, the loss of data is large, so a single through the histogram matching seems a little overwhelmed.

based on the implementation of the OPENCV, I encapsulated it as a function (the input and output are grayscale images, and the grayscale level is set to 8, and 0-255), it is necessary to directly take the use of

<span style= "FONT-SIZE:18PX;" ><span style= "FONT-SIZE:18PX;"  >double gethistsimilarity (const mat& I1, const mat& I2) {int histsize = 256;float range[] = {0,256};const float* Histrange = {Range};bool uniform = true;bool accumulate = false; Mat hist1,hist2;calchist (&i1,1,0,mat (), hist1,1,&histsize,&histrange,uniform,accumulate); normalize ( Hist1,hist1,0,1,norm_minmax,-1,mat ()); Calchist (&i2,1,0,mat (), Hist2,1,&histsize,&histrange,uniform, accumulate); normalize (Hist2,hist2,0,1,norm_minmax,-1,mat ()); return Comparehist (Hist1, Hist2, Cv_comp_correl);} </span></span>

The methods of histogram comparison are implemented in OPENCV:

/* Histogram comparison methods */
Enum
{
Cv_comp_correl = 0,
CV_COMP_CHISQR = 1,
Cv_comp_intersect = 2,
Cv_comp_bhattacharyya = 3,
Cv_comp_hellinger =cv_comp_bhattacharyya
};
are: correlation, chi-square, intersection coefficient and Babbitt distance, the calculation formula is as follows:

Some of the improvement of histogram method is to improve the histogram can not take advantage of spatial location information weaknesses, such as the fragtrack algorithm, the image is then split into a horizontal patch, and then for each patch search and matching histogram, to calculate the similarity of two images, Thus, the histogram corresponding to the location information, but the computational efficiency is not high.

the method of matrix decomposition

Method Description: The image patch is decomposed by matrix, such as SVD singular value decomposition and nmf nonnegative matrix decomposition, and then the similarity calculation is done.

Method thought: Because the image itself is a matrix, we can rely on matrix decomposition to obtain some more robust features to calculate the similarity of the image.

The advantages of SVD decomposition are: Singular value stability, proportional invariance, rotational invariance and compressibility. Singular value decomposition is based on the whole representation, not only has orthogonal transformation, rotation, displacement, image mapping and other algebraic and geometric invariance, but also has good stability and anti-noise, widely used in pattern recognition and image analysis. The purpose of singular value decomposition is to obtain a unique and stable feature description, reduce the dimension of the feature space and improve the anti-jamming ability.

The disadvantage of SVD decomposition is that there are negative numbers in the singular vectors which are decomposed by singular value decomposition, which can not explain its physical meaning well.

based on the method of NMF decomposition: the non-negative matrix is decomposed into the matrix and coefficient matrix which can embody the main information of the image, and can give a good explanation to the base matrix, such as the segmentation of human face, the base vector is the main concept characteristic of human "eyes" and "nose", and the source image is a weighted combination of the base Therefore, NMF plays a great role in the face recognition situation. There are many methods based on matrix eigenvalue calculation, such as trace transform, invariant moment calculation and so on.
based on feature point methodmethod Description: Statistics the number of matching feature points in two image patches, if the similarity of the maximum number of feature points, it is considered the most similar, the most matchingmethod Idea: The image can be described in feature points, such as Sift feature points, the corner points in the LK optical flow method and so on. So the measurement of similarity is changed to match the feature point. before doing some experiments, about the feature points matching, the affine transformation of an image, and then matching the characteristics between the points, the selected feature points have sift and fast sift deformation version surf. method Advantages: Can be selected as the characteristics of the general to meet the invariance, scale invariance, rotation unchanged. The similarity calculation of the image also has these invariance. The disadvantage of the method is that the matching calculation speed of the feature points is relatively slow, and the feature points may also have the wrong matching phenomenon.

a method based on peak signal-to-noise ratio (PSNR)

When we want to check the subtle differences in compressed video, we need to build a system that compares differences in video by frame. Most
The common comparison algorithm is Psnr (Peak signal-to-noise ratio). This is the simplest way to use "local mean error" to determine the difference, assuming that there are two images: I1 and I2, the number of rows is i,j, and there is a C channel. The value of each channel of each pixel occupies one byte, the range of values [0,255]. Note that when the two images are the same, the MSE value becomes 0. This causes the PSNR formula to be divided by 0 and becomes meaningless. So we need to deal with this particular situation separately. In addition, because of the wide dynamic range of pixels, logarithmic transformations are used to narrow the range when processing.

The Psnr method is implemented based on OPENCV, which is also encapsulated as a function, which can be called directly:

<span style= "FONT-SIZE:18PX;" >double Getpsnr (const mat& I1, const mat& I2) {Mat S1;absdiff (I1, I2, S1);       // | I1-i2|s1.convertto (S1, cv_32f);  Cannot make a square on 8 Bitss1 = S1.mul (S1);           // | I1-i2|^2scalar s = SUM (S1);         Sum elements per channeldouble SSE = S.val[0] + s.val[1] + s.val[2]; Sum Channelsif (SSE <= 1e-10)//For small values return Zeroreturn 0;else{double  MSE =sse/(double) (i1.channels () * I1.total ());d ouble Psnr = 10.0*log10 ((255*255)/mse); return Psnr;}} </span>

in the When examining the compressed video, the value is approximately between 30 and 50, and the larger the number, the better the compression quality. If the image difference is obvious, you may get a value of 15 or lower. The PSNR algorithm is simple and the check speed is very fast. But the difference in its appearance is sometimes disproportionate to a person's subjective feelings. So there is another algorithm called structural similarity to make this improvement.

structural similarity (SSIM) index measurement )
The structural similarity theory holds that the natural image signal is highly structured, that is, there is a strong correlation between pixels, especially the closest pixel in the airspace, which contains important information about the structure of objects in the visual scene; the main function of Hvs is to extract the structure information from the field of view, The measurement of structure information can be used to approximate the quality of image perception. The structure similarity theory is a new idea that differs from the previous low-order structure of HVS, and the biggest difference is the difference between top-down and bottom-up compared with the method based on HVS characteristics. The key to this new idea is the shift from the measurement of perceived error to the measurement of perceived structural distortion. It does not attempt to estimate the image quality by accumulating the errors associated with the simple cognitive model of mental physics, but directly estimates the structural changes of two complex structural signals, which in turn bypass the problem of the complexity of the natural image content and the multi-channel correlation. As the structure similarity theory realization, the structure similarity index from the image composition angle defines the structure information to be independent from the brightness, the contrast, reflects the scene object structure the attribute, and the distortion modelling is the brightness, the contrast and the structure three different factors combination. With the mean value as an estimate of brightness, The standard deviation is an estimate of the contrast, and the covariance As a measure of structural similarity.

The Ssim method is implemented based on OPENCV, which is also encapsulated as a function, which can be called directly:

<span style= "FONT-SIZE:18PX;" >scalar Getmssim (const mat& i1, const mat& i2) {Const double C1 = 6.5025, C2 = 58.5225;/*********************** Inits **********************************/int d = cv_32f;           Mat I1, I2;i1.convertto (I1, D); Cannot calculate on one byte large Valuesi2.convertto (I2, D);        Mat i2_2 = I2.mul (I2);        I2^2mat i1_2 = I1.mul (I1);        I1^2mat I1_i2 = I1.mul (I2);   I1 * i2/*************************** END inits **********************************/mat mu1, MU2; Preliminary Computinggaussianblur (I1, MU1, Size (11, 11), 1.5); Gaussianblur (I2, MU2, Size (11, 11), 1.5); Mat mu1_2 = Mu1.mul (MU1); Mat mu2_2 = Mu2.mul (MU2); Mat MU1_MU2 = Mu1.mul (MU2); Mat sigma1_2, sigma2_2, Sigma12; Gaussianblur (I1_2, Sigma1_2, Size (one, one), 1.5); sigma1_2-= mu1_2; Gaussianblur (I2_2, Sigma2_2, Size (one, one), 1.5); sigma2_2-= mu2_2; Gaussianblur (I1_i2, Sigma12, Size (one, one), 1.5); Sigma12-= mu1_mu2;/////////////////////////////////formuLA////////////////////////////////mat T1, t2, t3;t1 = 2 * mu1_mu2 + c1;t2 = 2 * sigma12 + c2;t3 = T1.mul (T2);               T3 = ((2*MU1_MU2 + C1). * (2*sigma12 + C2)) T1 = mu1_2 + mu2_2 + c1;t2 = sigma1_2 + sigma2_2 + c2;t1 = T1.mul (T2);      T1 = ((mu1_2 + mu2_2 + C1). * (sigma1_2 + sigma2_2 + C2)) Mat ssim_map;divide (T3, T1, Ssim_map); Ssim_map = T3./T1; Scalar MsSIM = mean (SSIM_MAP); MsSIM = average of Ssim Mapreturn MsSIM;} </span>

image Template Matching

In general, the source image and template image patch size, you can directly use the image similarity measurement method described above; if the source image is not the same size as the template image, a sliding matching window is often required to sweep the entire image to get the best matching patches.

The corresponding function in OpenCV is: Matchtemplate (): The function function is to slide the window in the input image to look for the similarity of each position to the template image patch. Similarity evaluation criteria (matching method) are: Cv_tm_sqdiff squared difference Matching method (the higher the similarity, the smaller the value), the Cv_tm_ccorr correlation matching method (multiplication operation, the higher the similarity is greater), the Cv_tm_ccoeff correlation coefficient matching method (1 means the best match,- 1 indicates the worst match).

Cv_tm_sqdiff Calculation formula:

Cv_tm_ccorr Calculation formula:

There is a new method for calculating similarity or for distance measurement:emd,earth Mover ' s distances

EMD is defined as the minimal cost this must be paid to transform one histograminto the other, where there is a "ground di Stance "Between the basic featuresthat is aggregated into the histogram.

Light changes can cause the image color values to drift, although the drift does not change the shape of the color histogram, but the drift caused the color value position changes, which may lead to the expiration of the matching strategy. The EMD is a metric that measures how to transform one histogram into the shape of another histogram, including moving a section (or all) of the histogram to a new location, which can be measured on a histogram of any dimension.

there are corresponding calculation methods in OpenCV: CvCalcEMD2 (). Based on the OPENCV Support Library, the EMD distance between histogram equalization and the HSV color space histogram of the original image is calculated.
*********************************************************************************************************** ********************************************************************************************

2015-7-24

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Summary of image similarity measurement and template matching