About Image Feature extraction

Feature extraction is a concept in computer vision and image processing. It refers to the use of the computer to extract image information, determine whether each image point belongs to an image feature. The result of feature extraction is that the points on the image are divided into subsets, which often belong to isolated points, continuous curves or contiguous regions.

Definition of a feature

So far there is no universal and precise definition of features. The exact definition of a feature is often determined by the problem or application type. Feature is a "fun" part of a digital image, which is the starting point for many computer image analysis algorithms. The success of an algorithm is therefore often determined by the characteristics it uses and defines. Therefore, one of the most important features of feature extraction is "repeatability": The features extracted from different images in the same scene should be the same.

Feature extraction is a primary operation in image processing, which means that it is the first arithmetic processing for an image. It examines each pixel to determine whether the pixel represents a feature. If it is part of a larger algorithm, then the algorithm generally examines only the feature area of the image. As a precondition operation of feature extraction, the input image is generally smoothed in the scale space by Gaussian fuzzy kernel. One or more features of an image are then computed by local derivative operations.

Sometimes, if feature extraction requires a lot of computational time, and the time limit can be used, a high-level algorithm can be used to control the feature extraction hierarchy, so that only part of the image is used to look for features.

Because many computer image algorithms use feature extraction as their primary calculation step, a large number of feature extraction algorithms are developed, and their extraction features are varied, and their computational complexity and repeatability are very different.


Edge
Edges are pixels that make up the boundaries (or edges) between two image regions. Generally, an edge shape can be arbitrary, and may include intersections. In practice the edges are generally defined as subsets of points that have large gradients in the image. Some commonly used algorithms also link high-gradient points to form a more perfect edge description. These algorithms may also impose some limitations on edges.

A partial view of the edge is a one-dimensional structure.


Angle
The corner is a point-like feature of the image, which has a two-dimensional structure locally. The early algorithms first perform edge detection and then analyze the edge's direction to find the edge abrupt turn (angle). The algorithm developed later eliminates the need for edge detection, but is able to look for height curvature directly in the image gradient. It was later found that it was sometimes possible to find areas with the same angular features in the image where there were no corners.


Regional
Unlike angles, regions describe a regional structure in an image, but a region may also consist of only one pixel, so many area detections can also be used to monitor angles. An area monitor detects an area in the image that is smooth for the angle monitor. Area detection can be imagined to shrink an image and then perform angular detection on a zoomed-out image.


Ridge
Long-shaped objects are called ridges. In practice, ridges can be thought of as a one-dimensional curve representing a symmetric axis, in addition to a ridge width that is localized for each ridge pixel. Extracting ridges from gray gradient images is more difficult than extracting edges, angles and regions. Ridge detection is often used in aerial photography to identify roads, which are used in medical images to identify blood vessels.


Feature Extraction
The feature can be extracted from the image after it is detected. This process may require many image-processing computers. The result is called a feature description or eigenvector.


Commonly used image features are color features, texture features, shape characteristics, spatial relationship characteristics.

A color feature

(a) Characteristics: color characteristics is a global feature, describes the image or image area corresponding to the surface properties of the scene. The general color feature is a pixel-based feature, where all pixels belonging to an image or image region have their own contribution. Because the color is not sensitive to the direction and size of the image or image area, the color feature does not capture the local characteristics of the object in the image well. In addition, when using only color feature queries, many unwanted images are often retrieved if the database is large. Color histogram is the most commonly used method of expressing color characteristics, its advantage is not affected by the change of image rotation and peace, the further use of normalization can not be affected by the image scale changes, the base disadvantage is not to express the color space distribution information.

(ii) commonly used feature extraction and matching methods

(1) Color histogram

The advantage is that it can simply describe the global distribution of colors in an image, that is, the proportion of different colors in the whole image, especially for those images that are difficult to automatically segment and those that do not need to take into account the space position of the object. The disadvantage is that it cannot describe the local distribution of the colors in the image and the spatial location of each color, that is, it is impossible to describe a specific object or an object in the image.

Most commonly used color space: RGB color space, HSV color space.

Color histogram feature matching method: Histogram intersection method, distance method, center distance method, reference color table method, additive color histogram method.

(2) Color set

Color Histogram method is a global color feature extraction and matching method, can not distinguish the local color information. A color set is an approximation of a color histogram that first transforms an image from an RGB color space into a visually balanced color space (such as the HSV space) and quantifies the color space into several handles. Then, the image is divided into several regions by the color auto-segmentation technique, and each region is indexed by a color component of the quantization color space, thus expressing the image as a binary color index set. Comparing the distance between different image color sets and the spatial relationship of color regions in image matching

(3) Color moment

The mathematical basis of this method is that any color distribution in the image can be represented by its moment. In addition, since the color distribution information is mainly concentrated in the low-order moment, only the first-order moment (mean), second moment (variance) and third-order moment (skewness) of the color can be used to express the color distribution of the image.

(4) Color aggregation vector

The core idea is to divide the pixels belonging to the histogram into two parts, if some pixels within the handle occupy a contiguous area larger than the given threshold, the pixels within that region are aggregated pixels, otherwise they are non-aggregated pixels.

(5) Color correlation diagram

Two texture features

(a) Features: Texture features are also a global feature, it also describes the image or image area corresponding to the surface properties of the scene. But because the texture is only an object surface characteristics, and does not fully reflect the nature of the object, so only the use of texture features can not obtain high-level image content. Unlike color features, texture features are not pixel-based, and they need to be statistically calculated in areas that contain multiple pixel points. In pattern matching, this kind of regional characteristics has great superiority, and does not match successfully because of local deviation. As a statistical feature, texture features often have rotational invariance and have strong resistance to noise. However, texture features also have its drawbacks, and one obvious drawback is that when the resolution of the image changes, the computed texture may have a large deviation. In addition, because it is possible to be affected by light, reflection, the texture reflected from the 2-d image is not necessarily the real texture of the surface of the three-to-three object.

For example, the reflection in water, the effect of smooth metal surfaces reflecting each other, can lead to changes in texture. Since these are not properties of the object itself, the texture information is applied to the retrieval, and sometimes these false textures can cause "misleading" retrieval.

It is an effective method to retrieve texture images with large differences in thickness, density and so on. But when the texture between the thickness, density and other easy to distinguish between the information is not small, the usual texture features are difficult to accurately reflect the human visual perception of different textures between the differences.

(ii) commonly used feature extraction and matching methods

Classification of texture feature description methods

(1) The typical representative of statistical methods is a texture feature analysis method called Gray-scale co-occurrence matrix Gotlieb and Kreyszig, on the basis of studying the various statistical features of the Symbiosis Matrix, the paper obtains four key features of the gray-scale symbiosis matrix through experiments: energy, inertia, entropy and relativity. Another typical method of statistical method is to extract the texture feature from the image's autocorrelation function (that is, the energy spectrum function of the image), that is, by calculating the energy spectrum function of the image, extracting the characteristic parameters of texture's thickness and directivity.

(2) Geometrical method

The so-called geometric method is a texture feature analysis method based on the theory of texture primitives (basic texture elements). The texture primitive theory holds that the complex texture can be made up of several simple texture primitives in a certain regular form. In the geometric method, there are two kinds of algorithms that have influence: Voronio Checkerboard Feature method and structure method.

(3) Model method

The model method is based on the image construction model, and the parameters of the model are used as the texture feature. Typical methods are random-airport model methods, such as Markov (Markov) random field (MRF) model method and Gibbs random airport model method.

(4) Signal processing method

Texture feature extraction and matching mainly include: gray-scale symbiosis matrix, Tamura texture feature, autoregressive texture model, wavelet transform, etc.

The feature extraction and matching of gray-scale symbiosis matrix mainly depends on four parameters: energy, inertia, entropy and correlation. Tamura texture features based on the human visual perception of the texture of psychological research, proposed 6 kinds of properties, namely: roughness, contrast, direction, line image, regularity and roughness. The autoregressive texture model (simultaneous auto-regressive, SAR) is an application example of Markov random field (MRF) model.

Three shape features

(a) Features: All kinds of search methods based on Shape feature can be used to retrieve the objects of interest in the image, but they also have some common problems, including: ① the current shape-based retrieval method lacks the perfect mathematic model. ② If the target is deformed, the retrieval results are often unreliable. ③ Many shape features describe only the properties of the target area, and it is often necessary to describe the target in terms of time and storage; ④ many shape characteristics reflect the target shape information is not exactly the same as the human intuition, or the similarity of feature space is different from that of human visual system. In addition, the three-dimensional object from the 2-d image is actually a projection of the object in the space of a plane, the shape reflected from the 2-d image is often not the real shape of the three-dimensional object, due to changes in the viewpoint, it may produce various distortions.

(ii) commonly used feature extraction and matching methods

Several typical methods for describing the shape characteristics of Ⅰ

Generally, there are two kinds of representation of shape features, one is contour feature and the other is region feature. The contour feature of the image is mainly aimed at the outer boundary of the object, while the region characteristic of the image relates to the whole shape area.

Several typical methods of describing shape features:

(1) Boundary feature method This method obtains the shape parameter of the image by describing the boundary feature. The method of detecting parallel straight line and boundary direction histogram by Hough transform is a classical method. Hough transform is a method of connecting edge pixels to form an area closed boundary by using the global characteristics of the image, and its basic idea is the duality of the point-line; the boundary direction histogram method first obtains the image edge by the differential image, then makes the histogram about the edge size and direction. The usual method is to construct the image gray gradient direction matrix.

(2) Fourier shape Descriptor method

The basic idea of the Fourier shape descriptor (Fourier shape deors) is to use the Fourier transform of the object boundary as the shape description, using the sealing and periodicity of the region boundary to transform the two-dimensional problem into one-dimensional problem.

Three kinds of shape expressions are derived from boundary points, namely curvature function, centroid distance and complex coordinate function.

(3) Geometrical parameter method

The expression and matching of shapes are described using a more simple method of regional characterization, such as shape factor, which is based on shape quantitative measures (such as moment, area, perimeter, etc.). In the QBIC system, the geometric parameters such as roundness, eccentricity, spindle direction and algebraic invariant moment are used for image retrieval based on shape feature.

It is necessary to explain that the extraction of shape parameters must be based on image processing and image segmentation, the accuracy of parameters is bound to be affected by the segmentation effect, the image with poor segmentation effect, shape parameters can not even be extracted.

(4) Shape invariant moment method

The moment of the area occupied by the target is used as the shape description parameter.

(5) Other methods

In recent years, the work on the representation and matching of shapes also includes methods such as finite element method (finite element or FEM), rotation function (Turning), and wavelet descriptor (Wavelet deor).

Ⅱ shape feature extraction and matching based on wavelet and relative moments

The method first uses the wavelet transform modulus to be extremely worthy to the multiscale edge image, then calculates 7 invariant moment of each scale, then transforms to 10 relative moment, the relative moment of all scales as the image characteristic vector, thus unifies the region and the closed, the non-closed structure.

Four spatial relationship characteristics

(a) Characteristics: the so-called spatial relationship, refers to the image of the multiple objects separated from each other space position or relative direction relationship, these relationships can also be divided into connection/adjacency relations, overlapping/overlap relations and inclusion/containment relations. Generally, spatial location information can be divided into two categories: relative spatial location information and absolute spatial location information. The former relationship emphasizes the relative situation between the targets, such as the upper and lower relations, and so on, the latter relationship emphasizes the distance between the target and the azimuth. It is obvious that the relative spatial position can be introduced by the absolute space position, but the relative spatial position information is often relatively simple.

The use of spatial relation features can enhance the ability to describe the content of images, but the characteristics of spatial relations are often sensitive to the rotation, inversion and scale change of image or target. In addition, the use of spatial information is often not enough, and can not effectively and accurately express the scene information. In order to retrieve, besides the use of spatial relationship features, other features are needed to match.

(ii) commonly used feature extraction and matching methods

There are two methods for extracting image spatial relationship features: one is to automatically segment the image, divide the object or color area in the image, and then extract the image feature from these regions and index it, and the other method simply divides the image evenly into several rules sub-blocks, The feature is then extracted for each image sub-block, and the index is established.
The problem of attitude estimation is to determine the azimuth direction problem of a three-dimensional target object. Attitude estimation has been applied in many fields such as robot vision, motion tracking and single camera calibration.

Sensors used in different fields for attitude estimation are not the same, and here the main talk is based on the visual attitude estimation.

The vision-based attitude estimation can be divided into monocular vision attitude estimation and multi-vision attitude estimation according to the number of cameras used. According to the different algorithm, it can be divided into model-based attitude estimation and learning-based attitude estimation.

A model-based attitude estimation method

Model-based methods are often used to estimate the geometric relationship of an object or the characteristic point of an object. The basic idea is to use a geometric model or structure to represent the structure and shape of an object, and to establish a correspondence between the model and the image by extracting some object features, and then using geometry or other methods to realize the estimation of the object's space attitude. The model used here may be a simple geometric form, such as a plane, a cylinder, a geometric structure, or a three-dimensional model obtained by laser scanning or other methods.

The model-based attitude estimation method is to update the attitude of the object by comparison with the real image and the synthetic image. In order to avoid optimizing search in global state space, the model-based method is generally used to decompose the optimization problem into multiple local features, which is very dependent on the accurate detection of local features. The robustness of this method is greatly affected when the noise is large and the exact local features cannot be extracted.

A method of attitude estimation based on learning

Based on the learning method, this paper studies the correspondence between two-dimensional observation and three-dimensional attitude from the training samples obtained in the different postures, and applies the learning decision rule or regression function to the sample, and the result is used as the attitude estimation of the sample. The study-based method generally adopts the global observation feature, which does not need to detect or recognize the local features of objects, and has better robustness. The disadvantage is that the precision and continuity of attitude estimation cannot be guaranteed because the dense sampling required for continuous estimation in high dimensional space cannot be obtained.

The attitude estimation method based on learning is derived from the idea of attitude recognition method. Attitude recognition needs to define a plurality of attitude categories, each of which contains a certain range of attitude, then the training samples are labeled for each attitude category, and the attitude classifier is trained by pattern classification to realize attitude recognition.

This kind of method does not need to model objects, generally through the image of the global characteristics of the matching analysis, can effectively avoid the local feature method in the complex posture and occlusion in the case of the feature matching ambiguity problem. However, the attitude recognition method can only divide the attitude into a number of pre-defined attitude categories, and the attitude can not be continuously and accurately estimated.

The study-based method generally adopts the global observation feature, which can ensure the algorithm has good robustness. However, the accuracy of the attitude estimation of this kind of method depends on the adequacy of training to a great extent. In order to obtain the correspondence between the two-dimensional observation and the three-dimensional attitude, it is necessary to obtain enough dense samples to learn the decision rules and regression functions. In general, the number of samples required is increased with the dimensionality of the state space, and for high dimensional state space, it is virtually impossible to obtain the dense sampling required for accurate estimation. Therefore, the lack of dense sampling and the difficulty in guaranteeing the accuracy and continuity of the estimation are the fundamental difficulties that can not be overcome by the attitude estimation method based on learning.

In contrast to typical pattern classification problems such as attitude recognition, the attitude estimation output is a high-dimensional attitude vector, rather than a class-label of a category. Therefore, this kind of method needs to learn a mapping from high-dimensional observation vectors to high-dimensional attitude vectors, which is still a very difficult problem in machine learning field at present.

Features are the best way to describe patterns, and we generally think that each dimension of a feature can describe a pattern from a different perspective, and ideally, the dimensions are complementary and complete.

The main purpose of feature extraction is dimensionality reduction. The main idea of feature extraction is to project the original sample into a low-dimensional feature space, and get the low-dimensional sample characteristics which can best reflect the sample essence or sample.

General image features can be divided into four categories: visual characteristics, gray statistical features, transformation coefficient characteristics and algebraic characteristics.

The visual characteristic mainly refers to the geometrical characteristic, the geometrical characteristic is stable, is affected by the human face posture change and the illumination condition and so on factors influence small, but is not easy to extract, moreover the measurement precision is not high, and the image processing technology is closely related.

Algebraic features are based on the characteristics of statistical learning method extraction. Algebraic features have high recognition accuracy, and algebraic feature extraction methods can be divided into two categories: one is the linear projection feature extraction method, the other is a nonlinear feature extraction method.

Traditionally, the feature extraction method based on principal component analysis and Fisher linear discriminant analysis is referred to as linear projection analysis.

Based on the feature extraction method of linear projection analysis, the basic idea is to find a linear transformation according to certain performance targets, compress the raw signal data into a low subspace space, make the distribution of data in sub-space more compact, provide a means for better description of data, and reduce the complexity of computation greatly. In linear projection analysis, the most representative of principal component analysis (PCA, or K-L transform) and Fisher linear discriminant Analysis (LDA), the feature extraction algorithm formed around these two methods has become the most classical and widely used method in the field of pattern recognition.

The main disadvantage of the linear projection analysis method is that a large number of existing samples need to be studied, and the location, illumination and non-linear deformation of the object are sensitive, so the acquisition condition has great influence on the recognition performance.

The method of nonlinear feature extraction is also one of the hotspots of research. "Nuclear techniques" were first used in SVM, and KPCA and KFA were the application of "nuclear techniques".

The basic idea of the nuclear projection method is to transform the sample in the original sample space into a space of high dimension and even infinite dimension through some form of nonlinear mapping, and solve it by using the linear analytic method in the new space with the aid of nuclear technique. Because the linear direction in the new space also corresponds to the nonlinear direction of the original sample space, the projection direction based on the kernel projection analysis also corresponds to the nonlinear direction of the original sample space.

The nuclear projection method also has some weaknesses: the geometrical meaning is unclear, it is impossible to know what distribution pattern the sample becomes after the non-explicit mapping, the selection of parameters in the kernel function is not the corresponding criteria, most of them can only be selected by the empirical parameters; not suitable for many cases of training samples, because after the nuclear mapping, The dimension of the sample is equal to the number of training samples, if the number of training samples is large, the vector dimension after the kernel map will be very high, and will encounter the problem of computational amount.

As far as the field of application is concerned, KPCA is not widely used in PCA. If as a general dimensionality reduction KPCA is indeed better than the PCA effect, especially when the feature space is not the general European space is more obvious. PCA can learn a subspace through a large number of natural images, but KPCA cannot.

The transformation coefficients feature refers to the Fourier transform, the wavelet transform and so on, and then the coefficients are identified as the characteristics.

About Image Feature extraction