Summarization and application of local feature point detection algorithm for image

The research of image feature detection has been a long time, many methods of image feature detection, coupled with the deformation of various algorithms, it is difficult to fully understand in a short period of time, but the main feature detection algorithm principle of the study. Overall, image features can include color features, texture principal, shape features, and local feature points. The local characteristics of which have good stability, not susceptible to external environment interference, this article is also a summary of this knowledge.

This article is now (2015/1/30) only in the form of a first draft, listed the framework of the subject, there are many other places need to be added and modified, such as the new Kaze feature extraction method based on nonlinear scale space in 2013 and its improved akate. In terms of application, there will be some examples with actual code, especially the search and motion target tracking based on feature points.

1. Local feature points

Image feature Extraction is the premise of image analysis and image recognition, it is the most effective way to simplify the expression of high-dimensional image data, from the $m\times n\times3$ data matrix of an image, we can not see any information, so we must extract the key information from the image according to this data, Some basic components and their relationships.

Local feature point is the local expression of image feature, it can only be used in the local particularity of image, so it is only suitable for matching, retrieving and so on. It is not suitable for image comprehension. The latter is more concerned with some global features, such as color distribution, texture characteristics, the shape of the main object and so on. Global features are susceptible to environmental disturbances, such as illumination, rotation, and noise, all of which can affect global characteristics. In contrast, local feature points, often corresponding to some lines in the image of the intersection, the structure of light and shade changes, less interference.

And the spot and the corner point are two kinds of local feature points. Spots usually refer to areas with different colors and shades of gray, such as a tree or a house on a prairie. It is a region, so it is more noisy than the corner point, stability is better. The corner point is the intersection between the corners of the object and the lines in the image.

2. Spot detection principles and Examples 2.1 log and Doh

Speckle detection methods mainly include the use of high Slapras operator detection method (LOG), and the use of Pixel point Hessian matrix (second-order differential) and its determinant value (DOH).

The log method has been described in detail in the spot detection article. Because the Laplace nucleus of the Ivigos function is much like a speck, the convolution can be used to find the structure of the speckle in the image.

Doh method is to use the second order differential Hessian matrix of image points:

$ $H (L) = \begin{bmatrix}l_{xx}&l_{xy}\\l_{xy}&l_{yy}\end{bmatrix}$$

and the value of its determinant doh (determinant of Hessian):

$ $det = \sigma^4 (L_{xx} (X,y,\sigma) L_{yy} (X,y,\sigma)-l_{xy}^2 (x,y,\sigma)) $$

The value of the determinant of the Hessian matrix also reflects the structure information of the local image. Compared with log, Doh has a good inhibitory effect on the spots of slender structures in the image.

Whether it is log or DOH, they detect the spots in the image, and the steps can be divided into the following two stages:

1) use different $\sigma$ to generate $\left (\frac{\partial^2g}{\partial x^2}+\frac{\partial^2g}{\partial y^2}\right) $ or $\frac{\ Partial^2g}{\partial x^2},\frac{\partial^2g}{\partial y^2},\frac{\partial^2g}{\partial x\partial y}$ template, and convolution operation of the image;

2) search for the peak of log and Doh response in the image location space and scale space.

2.2 SIFT

Detailed Algorithm Description Reference: Description of key steps of the SIFT location algorithm

In 2004, Lowe improved the efficient scale invariant feature transformation algorithm (SIFT), used the convolution of the original image and Gaussian kernel to establish the scale space, and extracted the characteristic points of the scale invariance on the Gaussian difference space pyramid. The algorithm has certain affine invariance, view invariance, rotational invariance and illumination invariance, so it has been widely used in image feature improvement.

The algorithm can be summarized as three steps: 1) The construction of Gaussian difference Pyramid, 2) feature point search, 3) feature description.

In the first step, it constructs a pyramid structure with a linear relationship with the structure of the group and layer, allowing us to find feature points on a continuous Gaussian kernel scale. Its better than log is that it uses the first-order Gaussian difference to approximate the Gaussian Laplace kernel, greatly reducing the computational capacity.

In the second step of the feature point search, the main key step is the interpolation of the extremum point, because in the discrete space, the local extremum point may not be the true extremum point, the true pole planting point can fall in the gap of the discrete point. So we need to interpolate the position of these slots and then find the coordinates of the extremum points.

The second step is to remove the point of the edge effect, because it is not enough to ignore the point that the dog is not responding enough, the dog's value will be affected by the edge, the points on the edge, although not the spots, but its dog response is also strong. So we're going to remove this part of the point. We use the area across the edge to exhibit the greatest and smallest principal curvature along the edge and the vertical edge direction. So by calculating the ratio of the principal curvature at the feature point, we can distinguish whether it is on the edge. This point can be found in the interpretation of the Harris corner point of the method.

The last step is the feature description of the feature point. The direction of the characteristic points is the histogram statistic of the gradient direction of the point in the neighborhood of the feature point, the main direction of the characteristic point is selected in the histogram, and a auxiliary direction can be selected. When calculating feature vectors, it is necessary to rotate the local image in the main direction, and then the gradient histogram statistic (4x4x8) in the neighborhood.

2.3 SURF

Detailed Algorithm Description reference: 1. Surf algorithm and source code analysis, on the 2. Surf algorithm and source code analysis, the next

2006, Bay and ESS based on the idea of SIFT algorithm, the acceleration robust feature (SURF) is proposed, which is mainly aimed at the disadvantage of the SIFT algorithm is too slow and the computational amount is large, and uses the approximate Harr wavelet method to extract the feature points. This method is based on the Hessian determinant (DoH) of the Speckle feature detection method. By using integral image at different scales, the approximate harr wavelet can be calculated effectively, simplifying the construction of second-order differential template and making the efficiency of feature detection in scale space higher.

The surf algorithm simplifies the second-order differential template by using the box filter on the integral image, and constructs the Hessian matrix element value, which shortens the time of feature extraction and improves the efficiency. The surf algorithm detects each pixel at each scale, and its approximate constructed Hessian matrix and its determinant are divided into different values:

$ $H _{approx} =\begin{bmatrix}d_{xx} (\sigma) &d_{xy} (\sigma) \\d_{xy} (\sigma) &d_{yy} (\sigma) \end{bmatrix} $ $

$ $c (X,y,\sigma) = d_{xx}d_{yy}-(0.9d_{xy}) ^2$$

Where $d_{xx},d_{xy}$ and $d_{yy}$ are approximate convolution values obtained by using box filters. If $c (X,Y,\SIGMA) $ is greater than the set threshold, the pixel is determined to be the keyword. Then the SIFT algorithm is approximate, the non-maximal value is suppressed in the $3\times3\times3$ Pixel neighborhood centered on the key point, and finally the precise location of the surf feature points is accomplished by interpolating the speckle feature.

The description of the surf feature points is also the full use of the integral graph, using the Harr wavelet template in two directions to calculate the gradient, and then using a sector to the adjacent points of the gradient direction of statistics, to obtain the main direction of the feature points.

3. The principle and example of corner point detection

The method of corner detection is also very many, among which the representative algorithm is Harris algorithm and fast algorithm.

Both of these algorithms have been specifically written to describe their algorithmic principles. Harris corner Point and fast feature point detection.

3.1 Harris Corner Point feature extraction

Harris Corner Point Detection is a first order derivative matrix detection method based on image grayscale. The main idea of the detector is local self-similarity/autocorrelation, that is, the similarity of the image block in a local window and the image block in the window after the small movement in each direction.

In the neighborhood of pixel points, the derivative matrix describes the change of data signal. Assuming that the block area is moved in any direction in the neighborhood of the pixel point, the pixel at the change point is the corner point if the intensity has changed drastically. The Harris matrix that defines $2\times2$ is:

$ $A (x) = \sum_{x,y}\omega (x, y) \begin{bmatrix}c_x^2 (×) &c_xc_y (x) \\C_xC_y (x) &c_y^2 (x) \end{bmatrix}= \begin{ bmatrix}a&b\\b&c\end{bmatrix}$$

Where, $C _x$ and $c_y$ are points $\mathbf{x} = (x, y) the first derivative of strength information in $x$ and $y$ directions, $\omega (x, y) $ is the weight of the corresponding position. The corner response value of the Harris matrix is calculated by D to determine whether it is an angle point. The formula is:

$ $D =deta-m (tracea) ^2= (ac-b) ^2-m (a+c) ^2$$

Where Det and trace are the operands of the determinant and trace, $m $ is a constant with a value of 0.04~0.06. When the corner response value is greater than the set threshold, and is the local maximum value in the neighborhood of the point, the point is treated as a corner point.

3.2 Fast angular point feature extraction

Fast algorithm based on accelerated segmentation test can quickly extract corner features. The algorithm determines whether a candidate point $p$ is a corner point, based on a pixel point $p$ as the center, the radius of 3 pixels of the discretized bresenllam circumference, under the condition of a given threshold $t$, if there is $n$ continuous pixel gray value in the circumference is greater than $i (p) +t$ or less than $i (p )-t$.

For the definition above, we can use a quick method to complete the detection without having to compare all the points on the circumference. First compare the pixel value relationship between the top and left four points, at least 3 points of the pixel gray value is greater than $i (p) +t$ or less than $i (p)-t$, then $p$ as a candidate, and then further complete judgment.

In order to speed up the algorithm, we can use machine learning ID3 greedy algorithm to construct decision tree. What needs to be explained here is that in 2010, Elmar and Gregory put forward the adaptive Universal Accelerated Segmentation Detection (agast) algorithm, by transforming the ID3 decision tree of fast algorithm into two-fork tree, and can allocate decision tree dynamically and efficiently according to the current processing image information. The arithmetic speed is improved.

4. Binary string feature descriptors

It can be noted that in the two corner detection algorithms, we do not refer to the description of feature points as sift or surf. In fact, once the feature points are detected, either the spot or the corner description method is the same, you can choose the most effective feature descriptors.

Feature description is an essential step for image matching and image searching. So far, people have studied a variety of feature descriptors, which are more representative of floating-point feature descriptors and binary cap string feature descriptors.

As in the SIFT and surf algorithms, the descriptors described by the gradient statistic histogram belong to the floating-point feature descriptor. But they calculate, the algorithm is complex, the efficiency is low, so later appeared many new characteristic description algorithm, like brief. Later a lot of binary string description sub-orb,brisk,freak and so on it is based on the improvement.

4.1 Brief algorithm

The main idea of the BRJEF algorithm is to select several pixel pairs in the neighborhood of the feature points, and compare the results of the comparisons into a binary string to describe the feature points by comparing the gray values of these points. Finally, the Hamming distance is used to calculate whether the feature descriptors match.

A detailed description of the brief algorithm can be consulted: Brief feature descriptors

4.2 Brisk algorithm

The brisk algorithm does not select Fast feature point detection in the feature point detection, but chooses the agast algorithm with stronger stability. In the construction of feature descriptors, the brisk algorithm can describe each feature point by using simple pixel gray value comparison, and then get a cascade binary bit string, which is consistent with the brief principle. In the brisk algorithm, the neighborhood sampling mode is used, that is, the discrete Bresenham concentric circles with different radii are constructed with the feature points as the center, then n sampling points with the same spacing are obtained on each concentric circle.

Since this neighborhood sampling mode can produce the effect of image grayscale aliasing in sampling, the brisk algorithm first makes a Gaussian smoothing image of the image. and the Gaussian function standard deviation $\sigma_i$ is proportional to the point spacing on each concentric circle.

Assuming that a pair of sample points $ (P_i,p_j) $ is arbitrarily selected in the $\dbinom{n}{2}$ sample point, the smoothed grayscale values are $i (p_i,\sigma_i) $ and $i (P_j,\sigma_j) $, the local gradient between two points is:

$ $g (P_i,p_j) = (p_j-p_i) \frac{i (P_j,\sigma_j)-I (p_i,\sigma_i)}{\left\|p_j-p_i\right\|^2}$$

Assuming that the set of all sampling point pairs is recorded as $\boldsymbol{a}$, the

$$\boldsymbol{a} = \{(p_i,p_j) \in r^2\times r^2 | i < N and j<i and I,j\in n\}$$

The set of sets S and long distance sampling points consisting of a short distance sample point pair are:

$ $S =\{(p_i,p_j) \in A | \left\|p_j-p_i\right\|<\delta_{max}\} \subseteq a$$

$ $L =\{(p_i,p_j) \in A | \left\|p_j-p_i\right\|>\delta_{min}\} \subseteq a$$

Where the distance threshold is usually set to $\delta_{max} = 9.75\delta,\delta_{min} = 13.67\delta$, where $\delta$ is the scale of the feature point.

Since long-distance sampling points contain more characteristic point angle information and local gradients cancel each other, the characteristic mode direction of the feature points can be calculated in set L:

$ $g = \begin{pmatrix}g_x\\g_y\end{pmatrix} = \frac{1}{l}\sum_{(P_i,p_j) \in l}g (P_i,p_j) $$

The sampling mode is then rotated around the feature point $\alpha = Arctan2 (g_y,g_x) $, and the feature descriptor has a rotational invariance.

Finally, a 512-bit binary string descriptor is formed by comparing all the feature points to $ (P_i^{\alpha},p_j^{\alpha}) $ line pixel gray values within the set S of the short distance sampling point after rotation.

4.3 Orb algorithm

The Orb algorithm uses fast to detect feature points, and then uses Breif to characterize the feature points, but we know that brief does not have the concept of feature point orientation, so the Orb introduces a direction calculation method on the brief basis, and uses greedy search algorithm on the selection of point pairs. Pick out some of the highly differentiated point pairs used to describe the binary string. A detailed description of the Orb algorithm can be found in: Orb feature point detection.

4.4 Freak algorithm

Fast Retina KeyPoint, the rapid retinal key point.

According to the principle of the retina point-to-sample, the middle is dense, the farther away from the center of the more sparse. and from coarse to fine construct descriptors, exhaustive greedy search find relevance small. 42 Senses Wild, 1000 pairs of points of the combination, find the top 512. These 512 are divided into 4 groups, the first 128 are less relevant, can represent coarse information, and later more and more refined. Match the time can first look at the first 16bytes, that is, the part of the fine information, if the distance is less than a certain threshold, and then continue, otherwise you do not have to look down.

5. Application of image Matching

The goal of image matching is to accurately determine the similarity between two images. The definition of similarity between images changes with different application requirements. For example, in object retrieval systems (to find images containing Abraham Lincoln's face), we think that different images of the same object are similar. In the object category retrieval system (to find images containing human faces), we think that objects of the same class are similar.

The application of local feature points is mainly in the first similarity, that is to say, we need to design an image matching algorithm to determine whether two images are images of the same object or scene. The ideal image matching algorithm should consider the similarity between two images of the same object to be very high, while the similarity between the images of two different objects is very low, as shown in.

Because of the difference in illumination, environment and angle, the image of the same object is different, as the image of two cars in the same angle, and the imaging is not the same. It is impossible to judge whether a car is the same class if we compare it directly with the image. It is necessary to extract the feature points and match the feature points.

What transformations are there in the image? In general, including the illumination changes and geometric changes, the change in illumination is the global or local color changes in the image, and the variety of geometric changes are more, can be translation, rotation, scale, affine, projection transformation and so on. Therefore, when we study local feature points, we require the characteristic points to be stable to these changes, and to have strong uniqueness, so that the image can be distinguished from other kinds of images, that is, the distance between classes is small and the distance between classes is large.

6. References

[1] Image feature extraction and matching algorithm based on corner point, Sickingron, 2014.

[2] image matching and recognition based on local features, Gomingming, 2014.

[3] Image feature extraction algorithm based on visual information, Daikimpo, 2014.

[4] Image local invariance characteristics and description, Wang Yongming, Wang Guijin authoring.

Summarization and application of local feature point detection algorithm for image