Surf of pattern matching----feature point detection learning _2 (surf algorithm)

In the previous blog feature point detection learning _1 (SIFT algorithm), the classical SIFT algorithm is introduced briefly, the SIFT algorithm is stable, the detected feature points are also more, the biggest determination is the high computational complexity. There are many scholars to improve it, in which the more famous is the surf algorithm introduced in this paper, the Chinese meaning of surf is fast robust feature. This article is not specifically about surf all the theories (the best theory is the author's thesis), but the surf algorithm is the next collation, convenient for later review.

Some articles on the internet have introduced surf, such as:

http://wuzizhang.blog.163.com/blog/static/78001208201138102648854/

Surf algorithm principle, there are some simple introduction.

http://blog.csdn.net/andkobe/article/details/5778739

Some of the details of surf are explained in an easy-to-understand.

http://www.owlei.com/DancingWind/

This article is called "Surf original Translation", written very well, after reading will be used in the surf some of the technology more in-depth understanding, but this article is not translated in English, but the author's own understanding of the integration map, Hessian matrix and other reasons for the introduction of a popular explanation, recommend a look.

　　　　First, surf description sub-formation steps

1. Constructing Gaussian pyramid scale space

In fact, the pyramid image of the surf structure is very different from the sift, because these differences have accelerated the speed of its detection. Sift uses the dog image, and surf uses the Hessian matrix determinant approximation image. First, take a look at the Hessian matrix of a pixel in the image, as follows:

That is, you can find a Hessian matrix for each pixel point. However, since our characteristic points need to be scale-independent, it is necessary to Gaussian filter before Hessian matrix is constructed. In this way, after filtering in the calculation of Hessian, its formula is as follows:

The symbols in the formula can be guessed by friends who are somewhat mathematically based, and there is not much to explain here.

Finally we want a transformed image of the original image, because we want to find the feature point on this transform image, and then map its position back to the original, for example, in sift, we are looking for feature points on the original dog graph. So what is this transformation diagram in surf? From Surf's numerous data, it is the approximate value of the Hessian matrix determinant of each pixel of the original image. The approximate formula of its determinant is as follows:

0.9 is an empirical value given by the author, in fact it is a set of theoretical calculations, specifically to see Surf's English paper.

As the Hessian is to be Gaussian smoothed first, and then second derivative, which in the discrete pixel is formed by the template convolution, the 2 operation together with a template instead of, for example, the template in the Y direction is as follows:

The left side of the graph uses Gaussian smoothing and a template for second derivative in the Y direction, so that the approximate processing is used to speed up the operation, and the results are as shown in the image on the right, which simplifies a lot. And the right image can be used to calculate the integral graph, greatly speeding up the speed, about the introduction of the integration map, you can go to the relevant information.

Similarly, the second-order mixed-offset template in the X and Y directions is as follows:

　　　　

The above-mentioned so much just got an approximate hessian determinant chart, this example than the sift in the dog chart, but in the pyramid image is divided into many layers, each layer is called a octave, each octave in a few different scales of the picture. In the SIFT algorithm, the image size (i.e. size) in the same octave layer is the same, but the scale (that is, the degree of ambiguity) is different, and the size of the picture in the different octave layers is not the same, because it is from the previous layer of the picture down sampling. When Gaussian Blur is performed, the Gaussian template size of sift is always constant, only changing the size of the image between different octave. In the surf, the size of the picture is constant, different octave layer to be detected by the image is to change the size of the Gaussian blur size obtained, of course, the same octave in the image of the Gaussian template scale is also different. Surf uses this method to save the reduction of the sampling process, and its processing speed is naturally put up. Its pyramid image is as follows:

　　

　　　　

　　2. Using non-maximal value to suppress the initial feature point determination

This step, similar to sift, compares each pixel that has been processed by the Hessian matrix to a size comparison of 26 points in its 3-dimensional field, and if it is the maximum or minimum value in these 26 points, it is preserved as a preliminary feature point.

　　3. Precise positioning of extreme points

Similar to the SIFT algorithm, the 3-dimensional linear interpolation method is used to obtain sub-pixel feature points, and also to remove those points less than a certain threshold value.

　　4. Select the main direction of the feature point.

This step differs greatly from the SIFT. Sift select the main direction of the feature point is the main direction of the feature point, which is to statistic its gradient histogram in the feature point domain, take the largest bin value of histogram and those directions exceeding the maximum bin value 80%. In surf, the gradient histogram is not counted, but the Harr wavelet feature in the domain of statistical feature points. That is, in the domain of the feature point (for example, a circle with a radius of 6s, S is the scale of the point), the sum of the horizontal Haar wavelet feature and the vertical Haar wavelet feature of all points in the 60-degree fan is counted, and the Haar wavelet's size becomes 4s, so a fan gets a value. Then the 60-degree fan rotates at a certain interval, and the direction of the fan in the maximum value is the main direction of the feature point. The process is as follows:

　　　　

　　5. Construct Surf feature point description operator

In sift, the neighborhood of 16*16 is taken around the feature point, and the field is transformed into a small area of 4*4, each small area is statistically 8 direction gradients, and finally a vector of 4*4*8=128 dimension is obtained, which acts as the SIFT descriptor for that point.

In surf, you also take a square box around the feature point, where the side length of the box is 20s (s is the scale at which the feature point is detected). The direction of the frame, of course, is the 4th step to detect the main direction. The box is then divided into 16 sub-regions, each of which counts the horizontal and vertical Haar wavelet characteristics of 25 pixels, where the horizontal and vertical directions are relative to the main direction. The Haar wavelet feature is the sum of the horizontal direction values, the sum of the horizontal direction absolute value, the sum of the vertical direction and the absolute value of the vertical direction. The procedure is as follows:

　　　　

So each small area has 4 values, so each feature point is the vector of the 16*4=64 dimension, less than half of the SIFT, which will greatly accelerate the matching speed during the feature matching process.

　　Two, the matching process of characteristic points

The matching process of the surf feature points is similar to the SIFT, which is not described in detail here.

　　Third, the experimental part

This experiment uses the online popular open surf, completes with the C + +, uses to the OpenCV library, is: http://www.chrisevansdev.com/computer-vision-opensurf.html

The author of the Code gives the main function to implement 6 functions, including static picture feature points detection, video feature point detection, image matching, video and image matching, feature points clustering and other 6 functions. This experiment is a simple test of the image detection, matching and feature points clustering 3 functions. And added a simple interface.

The development environment is: Opencv2.4.2+qt4.8.2+open surf+windosxp

The experiment is divided into the following 3 sections to describe.

　　Surf feature point detection and description

Open the software, click the Open Image button, select a picture to be detected, the effect is as follows:

　　

Click the Surf Detect button, the program will perform feature point detection of the image, and display feature results, including the main direction of the feature point, scale and other information. The effect is as follows:

　　

Click the Close button to exit the program.

　　Surf feature point Matching

Open the software, click the Open Image button, and then open 2 images to match, 2 pictures to have the same content, only the scale, rotation, lighting and other different. Open the picture as follows:

　　

Click the Surf Detect button, similar to the above, the image will be detected first, the effect is as follows:

　　

Click the Surf Match button and the program will match the detected image with the feature points, as follows:

　　

Click the Close button to exit the program.

　　Surf Feature point Clustering

Open the software, click the Open Image button, and select a picture to classify the feature points as follows:

　　

Click the Surf Detect button to first check the image for Surf feature points, as follows:

　　

Click the Kmeans cluster button and the program will cluster these feature point collections and display their results as follows:

　　

Click the Close button to exit the program.

　　Experiment main function part and code (appendix has project code download link):

Opensurf.h:

#ifndef Opensurf_h
#define Opensurf_h

#include <QDialog>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/calib3d/calib3d.hpp>
#include "Ipoint.h"
#include "Kmeans.h"

using namespace CV;

Namespace Ui {
Class Opensurf;
}

Class Opensurf:public Qdialog
{
Q_object

Public
Explicit Opensurf (Qwidget *parent = 0);
~opensurf ();

Private Slots:
void on_openbutton_clicked ();

void on_detectbutton_clicked ();

void on_matchbutton_clicked ();

void on_closebutton_clicked ();

void on_clusterbutton_clicked ();

Private
Ui::opensurf *ui;
Iplimage *img1, *img2, *img_match1, *IMG_MATCH2;
Ipvec ipts, Ipts1, ipts2;
Ippairvec matches;
Kmeans km;
int open_image_num;

};

#endif//Opensurf_h

Opensurf.cpp:

#include "opensurf.h"
#include "ui_opensurf.h"
#include <QtGui>
#include <QtCore>
#include "Surflib.h"

using namespace Std;

Opensurf::opensurf (Qwidget *parent):
Qdialog (parent),
UI (New Ui::opensurf)
{
Open_image_num = 0;
UI->SETUPUI (this);
}

Opensurf::~opensurf ()
{
Delete UI;
}

void Opensurf::on_openbutton_clicked ()
{
QString Img_name = Qfiledialog::getopenfilename (This, "Open Image", ".. /open_surf ",
TR ("Image Files (*.png *.jpeg *.jpg *.bmp)");
if (0 = = open_image_num)
Ui->textbrowser->clear ();
Open_image_num + +;
if (1 = = Open_image_num)
{
IMG1 = Cvloadimage (Img_name.toascii (). data ());
Img_match1 = Cvloadimage (Img_name.toascii (). data ());
Cvsaveimage (".. /open_surf/load_img1.jpg ", IMG1);
Ui->textbrowser->setfixedsize (Img1->width, img1->height);
Ui->textbrowser->inserthtml (" ");
}
else if (2 = = Open_image_num)
{
Img2 = Cvloadimage (Img_name.toascii (). data ());
IMG_MATCH2 = Cvloadimage (Img_name.toascii (). data ());
Cvsaveimage (".. /open_surf/load_img2.jpg ", IMG2);
Ui->textbrowser->setfixedsize (Img1->width+img2->width, Std::max (Img1->height, img2->height));
Remove the line wrap mode to display 2 images horizontally
Ui->textbrowser->setwordwrapmode (Qtextoption::nowrap);
Ui->textbrowser->inserthtml (" ");
}
else if (3 = = Open_image_num)
{
Open_image_num = 0;
Ui->textbrowser->clear ();
}
}

void opensurf::on_detectbutton_clicked ()
{
if (1 = = Open_image_num)
{
//Use Surf to detect feature points
Surfdetdes (IMG1, IPTs, False, 5, 4, 2, 0.0004f);
//Draw the feature point in the image
Drawipoints (IMG1, ipts);
Cvsaveimage (". /open_surf/detect_img1.jpg ", IMG1);
Ui->textbrowser->clear ();
Ui->textbrowser->setfixedsize (Img1->width, img1->height);
ui->textbrowser->inserthtml (" ");
}
Else if (2 = = Open_image_num)
{
//Detect feature points with surf
Surfdetdes (IMG1, Ipts1, False, 5, 4, 2, 0.0004f); //Draw the feature point in the image
Drawipoints (IMG1, ipts1);
Cvsaveimage (". /open_surf/detect_img1.jpg ", IMG1);
//Detect feature points with surf
Surfdetdes (Img2, Ipts2, False, 5, 4, 2, 0.0004f);
//Draw feature points out of the image
Drawipoints (Img2, ipts2) ;
Cvsaveimage (".. /open_surf/detect_img2.jpg ", IMG2);
Ui->textbrowser->clear ();
ui->textbrowser->inserthtml (" ");

Ui->textbrowser->setfixedsize (Img1->width+img2->width, Std::max (Img1->height, img2->height));
Remove the line wrap mode to display 2 images horizontally
Ui->textbrowser->setwordwrapmode (Qtextoption::nowrap);
Ui->textbrowser->inserthtml (" ");
}
}

void Opensurf::on_matchbutton_clicked ()
{
if (2 = = Open_image_num)
{
Getmatches (ipts1,ipts2,matches);
for (unsigned int i = 0; i < matches.size (); ++i)
{
Drawpoint (Img_match1,matches[i].first);
Drawpoint (Img_match2,matches[i].second);

Const int & w = img1->width;
Const INT & h1 = img1->height;
Const int & h2 = img2->height;
//Here because I already know the relative opening of the picture to show the position, so in the line to draw a point of common sense
//So this method is not universal, just for the picture given in this example, the best way is like Rob Hess's SIFT Algorithm
// Synthesize 2 pictures, then draw a matching line on a picture
Cvline (Img_match1,cvpoint (MATCHES[I].FIRST.X,MATCHES[I].FIRST.Y),
Cvpoint ( Matches[i].second.x+w,matches[i].second.y+std::abs (H1-H2)),
Cvscalar (255,255,255), 1);
Cvline (Img_match2,cvpoint (Matches[i].first.x-w,matches[i].first.y-std::abs (H1-H2)),
CvPoint (matches[i). SECOND.X,MATCHES[I].SECOND.Y),
Cvscalar (255,255,255), 1);
}
Cvsaveimage (".. /open_surf/match_img1.jpg ", img_match1);
Cvsaveimage (".. /open_surf/match_img2.jpg ", IMG_MATCH2);
Ui->textbrowser->clear ();
ui->textbrowser->inserthtml (" ");

Ui->textbrowser->setfixedsize (Img1->width+img2->width, Std::max (Img1->height, img2->height));
Remove the line wrap mode to display 2 images horizontally
Ui->textbrowser->setwordwrapmode (Qtextoption::nowrap);
Ui->textbrowser->inserthtml (" ");
}
}

void Opensurf::on_clusterbutton_clicked ()
{
for (int repeat = 0; repeat < ++repeat)
{

KM. Run (&ipts, 5, true);
Drawpoints (IMG1, km.clusters);

for (unsigned int i = 0; i < ipts.size (); ++i)
{
Cvline (IMG1, Cvpoint (IPTS[I].X,IPTS[I].Y), Cvpoint (km.clusters[ipts[i].clusterindex].x, Km.clusters[ipts[i]. CLUSTERINDEX].Y), Cvscalar (255,255,255));
}
Cvsaveimage (".. /open_surf/kmeans_img1.jpg ", IMG1);
Ui->textbrowser->clear ();
Ui->textbrowser->setfixedsize (Img1->width, img1->height);
Ui->textbrowser->inserthtml (" ");
}
}

void Opensurf::on_closebutton_clicked ()
{
Close ();
}

Summarize:

Surf is much faster than sift in speed, which is mainly due to its integration diagram technology, has reduced the use of the Hessian matrix reduction sampling process, in addition to the resulting feature vector dimension is also relatively small, facilitate the faster feature point matching.

Appendix I:

1. As with the Robhesson runtime, the following error occurs when the open surf is running:

IPOINT.OBJ:-1:ERROR:LNK2019: unresolved external symbol _cvfindhomography, the symbol in function "int __cdecl translatecorners (class std::vector< struct std::p air<class ipoint,class ipoint>,class std::allocator<struct std::p air<class Ipoint,class ipoint> > > &,struct cvpoint const * CONST,STRUCT cvpoint * const) "(Email protected]@[email protected][emai L protected]@@[email protected]@[email protected]@[email protected][email protected]@@[email protected]@[email Protected]@@[email protected]@[email protected]@[email protected]@[email protected]@z) is referenced in

But this time the reason is that there is no opencv_calib3d242d.lib library, because this open surf in the feature matching with the OpenCV in the 3-dimensional reconstruction of the function cvfindhomography (the function is to find the single-matrix between 2 images), So many people will forget to add this library file, it will cause this error.

2, if the use of Qt or MFC Interface design code, compile the program will report the following error:

moc_open_surf.obj:-1:error:lnk2005: "Public:void __thiscall kmeans::setipoints (class Std::vector<class Ipoint, Class Std::allocator<class ipoint> > *) "([email protected]@@[email protected]@@[email Protected]@@@[email Protected]@@[email protected]@@z) is already defined in Main.obj

In fact, the author of Open surf may not be thoughtful, it is in the Kmeans.h file Kmeans This class member function method implemented in the header file, in fact, this is not supported in standard C + +. The solution is to transform the kmeans.h into a kemans.hpp (the method I have not tried); another way is to create a new Kmeans.cpp file, the implementation of the member functions in the CPP file implementation, I this experiment is to use this method.

　　Appendix II:

Experimental Engineering code download.

Http://www.cnblogs.com/tornadomeet/archive/2012/08/17/2644903.html

Surf of pattern matching----feature point detection learning _2 (surf algorithm)