Image feature extraction of target detection (II.) LBP characteristics

LBP (local Binary pattern, partial two-value mode) is an operator used to describe the local texture feature of an image, and it has the notable advantages of rotational invariance and grayscale invariance. It is first by T. Ojala, M.pietikäinen, and D. Harwood were proposed in 1994 for Texture feature extraction. Moreover, the extracted feature is the local texture feature of the image;

1. Description of LBP features

The original LBP operator is defined in the window of the 3*3, with the window center pixel as the threshold value, the adjacent 8 pixels of the gray value of the comparison with its, if the surrounding pixel value is greater than the center pixel value, then the location of the pixel is marked 1, otherwise 0. In this way, 8 points in the 3*3 neighborhood can be compared to produce 8-bit binary numbers (typically converted to a decimal number, or LBP code, a total of 256), which is the LBP value of the center pixel of the window, and this value is used to reflect the texture information of the region. As shown in the following illustration:

an improved version of LBP:

After the original LBP was proposed, the researchers continued to propose various improvements and optimizations.

(1) Circular LBP operator:

The biggest drawback of the basic LBP operator is that it covers only a small area within a fixed radius, which obviously does not meet the needs of different sizes and frequency textures. In order to adapt to the texture characteristics of different scales, and to achieve the requirements of grayscale and rotational invariance, Ojala has improved the LBP operator, extended the 3x3 neighborhood to any neighborhood, and replaced the Square neighborhood with a circular neighborhood, and the improved LBP operator allows any number of pixels in a circular neighborhood with a radius of R. The LBP operator with P sampling points in a circular region with radius r is obtained.

(2) LBP rotation invariant mode

It can be seen from the definition of LBP that the LBP operator is invariant in grayscale, but not rotated. The rotation of the image will get different LBP values.

Maenpaa and other people have extended the LBP operator, and a LBP operator with rotational invariance is proposed, that is, a series of initial defined LBP values are obtained by rotating the circular neighborhood, taking its minimum value as the LBP value of the neighborhood.

Figure 2.5 shows a diagram of the process for obtaining a rotation-invariant LBP, the numbers below the operator indicate the LBP value of the operator, the 8 LBP patterns shown in the figure, the rotation-invariant processing, the resulting LBP value of rotation invariance is 15. In other words, the 8 LBP patterns in the graph correspond to a rotation-invariant LBP pattern of 00001111.

(3) LBP equivalent mode

A LBP operator can produce different binary modes, and the LBP operator with P sampling points in a circular region with radius r will produce P2 patterns. Obviously, with the increase in the number of sampling points in the neighborhood set, the type of binary mode is sharply increased. For example, 20 sample points within a 5x5 neighborhood, with 220 = 1,048,576 binary modes. So many of the two value patterns are not good for texture extraction or texture recognition, classification, and access to information. At the same time, too many pattern types are detrimental to the expression of textures. For example, when the LBP operator is used for texture classification or face recognition, the statistical histogram of LBP is used to express the information of the image, and more pattern types will make the data too large and the histogram is too sparse. Therefore, it is necessary to reduce the dimensionality of the original LBP pattern, which can best represent the information of the image when the data volume is reduced.

In order to solve the problem of too much binary mode and improve the statistic, Ojala proposes to use an "equivalence pattern" (Uniform pattern) to reduce the dimensionality of the pattern types of LBP operators. Ojala that the majority of LBP patterns in real images contain up to two jumps from 1 to 0 or 0 to 1. As a result, Ojala defines "equivalence mode" as the binary equivalent of a LBP that is referred to as an equivalence pattern class when the corresponding cyclic binary number from 0 to 1 or from 1 to 0 is changed by a maximum of two times. such as 00000000 (0 jumps), 00000111 (only one time from 0 to 1 of the jump), 10001111 (first from 1 to 0, then 0 to 1, a total of two jumps) are equivalent pattern classes. Patterns other than equivalence mode classes are classified as a mixed-mode class, such as 10010111 (a total of four jumps) (this is my personal understanding, not knowing right).

With this improvement, the type of binary mode is greatly reduced without any loss of information. The number of patterns is reduced from the original 2P species to P (P-1) + 2, where p represents the sample count in the neighborhood set. For 8 sample points in a 3x3 neighborhood, the binary mode is reduced from 256 to 58, which makes the eigenvectors less dimensional and reduces the impact of high-frequency noise.

2. The principle of LBP features for testing

It is obvious that the extracted LBP operator can get a LBP "code" at each pixel point, then, after extracting its original LBP operator for an image (the grayscale value of each pixel is recorded), the original LBP feature is still "a picture" (the LBP value of each pixel is recorded).

In the application of LBP, such as texture classification, face analysis and so on, the LBP map is not used as eigenvector for classification recognition, but the statistical histogram of LBP feature spectrum is used as feature vector for classification recognition.

Because, from the above analysis, we can see that this "feature" is closely related to the position information. Directly to two pictures to extract this "feature", and the discriminant analysis, will be because "position is not aligned" and produced a great error. Later, the researchers found that a picture could be divided into sub-regions, extracting LBP features for each pixel in each sub-region, and then establishing a statistical histogram of LBP features within each sub-region. As a result, each sub-region can be described by a statistical histogram, and the whole picture is composed of several statistical histograms;

For example, a 100*100 pixel-sized picture, divided into 10*10=100 sub-regions (which can be divided in several ways), each sub-region is 10*10 pixels, each pixel within each sub-region extracts its LBP feature, and then a statistical histogram is established; This picture has a 10*10 sub-area, there is a 10*10 statistical histogram, using this 10*10 statistical histogram, you can describe this picture. Then, we can judge the similarity between the two images by using various similarity measure functions.

3. The process of extracting LBP eigenvector

(1) First, the detection window is divided into 16x16 small area (cell);

(2) for one pixel in each cell, compare the gray values of the adjacent 8 pixels with their values, if the surrounding pixel value is greater than the center pixel value, the pixel position is marked as 1, otherwise 0. Thus, the 8 points in the 3*3 neighborhood can produce a 8-bit binary number, that is, the LBP value of the center pixel of the window is obtained;

(3) The histogram of each cell is then computed, that is, the frequency at which each number (assuming the value of the decimal number LBP) occurs, and the histogram is normalized.

(4) Finally, the resulting statistical histogram of each cell is connected to become a eigenvector, that is, the LBP texture feature vector of the whole picture;

You can then use SVM or other machine learning algorithms to classify it.

===========================================================================

LBP (Local Binary Pattern) for human face recognition

1. Introduction to Algorithms

LBP is a simple and effective feature extraction algorithm for texture classification. The LBP operator was put forward by Ojala and others in 1996, the main thesis is "Multiresolution gray-scale and rotation invariant texture classification with local Binary patterns ", Pami, vol, No.7, July 2002. LBP is the abbreviation for "Local binary pattern".

The explanation of the paper can be referred to the link http://blog.sina.com.cn/s/blog_916b71bb0100w043.html

From the point of view of texture analysis, the texture feature of a pixel on an image, in most cases, refers to the relationship between this point and the surrounding pixel point, that is, the relationship between the point and its neighboring points. from which angle the characteristics of this relationship are extracted, different kinds of characteristics are formed. With the feature, it can be categorized according to the texture. LBP constructs a relationship that measures a pixel point and its surrounding pixels.

For each pixel in the image, the gray value of the pixel is converted into a eight-bit binary sequence by calculating the size relationship of each pixel and the center pixel in the 3*3 neighborhood. The specific calculation process as shown below, for any point in the image of the IC, its LBP features calculated as the IC as the center, take the IC adjacent to the 8 points, in accordance with the clockwise direction of the I0,i1,..., I7; The pixel value of the IC point is the threshold value, if the pixel value of the Point II is less than IC, then II is 0 , otherwise 1; the two-valued 0-1 sequence is treated as a 8-bit binary number, and the binary number is converted to decimal to obtain the value of the LBP operator at the IC point.
The basic LBP operator is confined to the neighborhood of 3*3, so the researchers have extended the LBP operator for the large-scale structure of larger images, which is not very good at extracting the required texture features. The new LBP operator LBP (p,r) can calculate the different radius neighborhood size and the eigenvalues of different pixel points, where P represents the number of pixels around, R represents the neighborhood radius, and the original square neighborhood is extended to a circle, the following figure gives four kinds of extended LBP example, where R can be a decimal, For points that do not fall to an integer position, the gray value of the two-linear difference method can be computed based on the pixel gray value of the nearest two integer position in the orbit.

LBP (P,R) has 2^p values, that is, the image has a 2^p binary model, however, the actual study found that all modes of expression of the importance of information is different, statistical research shows that a few patterns in an image is particularly concentrated, to reach the total pattern of about 90% of the proportion, Ojala and others define this mode as uniform mode, if a binary sequence as a circle, 0-1 and 1-0 of the total number of occurrences of the sum of not more than two times, then this sequence is uniform mode, for example, 00000000, 00011110, 00100001, 11111111, when using LBP to express image textures, it is usually only concerned with the uniform pattern, and all other patterns are grouped into the same class.

The various LBP patterns of human face images are shown in the figure below, as can be seen from the graph, the change of the image and the original image can be more clearly reflected in the typical regions of the texture, but also to reduce the value of the study of the smooth area of the characteristics, while reducing the dimension of the feature. In comparison, the uniform mode is more realistic and is used in face recognition and expression recognition applications.

In the expression recognition, the most commonly used is the statistical histogram of LBP as the feature vector of the expression image. In order to consider the location information of the feature, the image is divided into several small areas, in each small area of the histogram statistics, that is, the number of the area belonging to a pattern, and finally the histogram of all the regions once joined together as a eigenvector to accept the next level of processing.

The LBP operator quantifies the point by using the relationship between the surrounding point and the point. The effect of illumination on image can be eliminated more effectively after quantization. As long as the illumination change is not enough to change the size relationship between the two point pixel values, the value of the LBP operator will not change, so to a certain extent, based on the LBP recognition algorithm solves the problem of illumination change, but when the image illumination changes unevenly, the size relationship between the pixels is destroyed, The corresponding LBP pattern also changed.

If the image is rotated, the texture feature rotates, and the resulting 2 binary string rotates, and the LBP value changes accordingly. In order for LBP to have rotational invariance, the binary string is rotated. Assuming that the LBP feature at the beginning is 10010000, the binary feature, rotated clockwise, can be converted to 00001001 form, so that the LBP value is minimal. Regardless of how the image is rotated, the minimum value of the binary feature extracted from the point is constant, with the minimum value as the LBP feature extracted, so that LBP is rotated unchanged. When P=8, the number of different binary features that can be produced is 2^8, which, after the above representation, becomes 36. (I thought it should be 2^8/8=32)