Basic concepts of image processing-convolution, filtering, smoothing (reprint)

/* Today, my brother asked me, CV's book is full of convolution, filtering, smoothing ... What exactly is the meaning of these concepts, what is the difference and connection, instant dizzy dish, learned so long CV, convolution, filtering, smoothing ... These concepts are repeated several times a day, but the heart also only understand a general meaning, hurriedly Google's ~ found that they know the real is not comprehensive, in this to do some summary, later on this basic concept to study deeply ~*/

1. Image convolution (template)

(1). Use templates to process image-related concepts:

Template: a matrix block whose mathematical meaning is a convolution operation.

      convolution operations: can be seen as a weighted summation process, Each pixel in the image area used corresponds to the phase   multiply of each element of the convolution nucleus (weight matrix), and the sum of all products is the new value of the center pixel of the region.
      convolution kernel: The weights used in convolution are represented by a matrix, which is a weight matrix.
      Convolution example:
              3 * 3  of the pixel area R with convolution kernel G:
               R5 (center pixel) =R1G1 + r2g2 + r3g3 + r4g4 + r5g5 + r6g6 + r7g7 + r8g8 + r9g9
       & nbsp;    

(2). Problems with using templates for image processing (cross-border issues):
Boundary problem: When processing image boundary pixels, convolution kernel and image use area cannot match, the center of convolution kernel and boundary pixel point correspond, the convolution operation will be problematic.
Treatment methods:
A Ignore the boundary pixels, that is, the processed image loses those pixels.
B Preserves the original boundary pixels, which is the copy boundary pixel to the processed image.

(3). Common templates:

Let's take a look at the concept of one-dimensional convolution.
The convolution definition of continuous space: the convolution of f (x) and g (x) is the integral value of f (t-x) g (x) in T from negative infinity to positive infinity. T-x to define a field in F (x), so it seems that a large amount of points is actually within a certain range. The actual process is f (x) to do a y-axis inversion, and then the x-axis to translate T is f (t-x), and then the G (x), the value of the product of the two integrals. Imagine if g (X) or F (x) is a unit of the order-function. So is the area of F (t-x) and g (X). This is the convolution.
Changing the integral symbol to sum is the convolution definition of the discrete space. What is the meaning of convolution in the image, that is, the image is the image f (x), the template is g (x), and then the model G (x) in the stencil move, each to a position, the F (x) and g (x) The definition field intersects the element convolution definition is often used for linear system analysis. The linear system is the linear relationship between the input and output of a system. This means that the whole system can be decomposed into N-many unrelated independent changes, and the whole system is the summation of these changes. such as X1->y1, x2->y2; So a*x1 + b*x2, A*y1 + b*y2 This is the linear system. Indicates that a linear system can be used in the form of integrals

,

That is, F (t,x) represents a linear coefficient such as a b. It looks like a convolution, right, if f (t,x) = f (t-x) is not it. From F (t,x) to F (t-x) is actually an indication that F (t,x) is a linear shift, that is, when the difference of the variable is not changed, Then the value of the function does not change. In fact, one thing is that the output of a linear shift-invariant system can be obtained by inputting and representing a function convolution of the linear characteristics of the system.

2. Image filtering

(1) Image filtering, that is, in the context of preserving the image detail characteristics of the target image noise suppression, is an indispensable operation of image preprocessing, the quality of its processing will directly ring to the subsequent image processing and analysis of the effectiveness and reliability. (filtering is to remove useless information, to retain useful information, may be low-frequency, may also be high-frequency)

(2) The purpose of filtering is two: one is to extract the characteristics of the object as a feature pattern of image recognition; The other is to adapt to the requirements of image processing, eliminating the noise that is mixed when the image is digitized.

There are two requirements for filtering processing: one is not to damage the image contour and edge of important information, and the second is to make the image clear visual effect is good.

(3) Image filtering method:

(a) Image filtering methods are many, mainly can be divided into frequency domain method and spatial domain method two categories. The processing of frequency domain method is to calculate the transformation coefficient value of image in some transform domain of image, then obtain the enhanced image by inverse transform. This is a kind of indirect image filtering method. The spatial filtering method is a kind of direct filtering method, which is used to calculate the image grayscale directly when processing the image.

<1> frequency Domain filtering: It is a method to convert images from space or time domain to frequency domain, and then use the transformation coefficients to reflect the properties of some image features for image filtering. Fourier transform is a kind of common transformation. In the Fourier transform domain, the DC component of the spectrum is proportional to the average brightness of the image, and the noise corresponds to the region with the higher frequency, and the image entity is located in the lower frequency region. These intrinsic properties of images in transformations can be used for image filtering. Can construct a low-pass filter, so that the low-frequency component smoothly through and effectively resistance to high-frequency components, you can filter out the image noise, and then through the inverse transformation to obtain a smooth image.

The mathematical expression for low pass is shown in the following formula:
               

The Fourier transform of the original image containing noise in the formula F (U, v);
H (U, v) is a transfer function, also known as a transfer function (i.e. low-pass filter);
G (U, v) is a Fourier transform of the output image after low-pass filtering.
The H filter filters out high-frequency components, and the low-frequency information is basically passed without loss. After filtering, the inverse transform of Fourier transform can get smooth image, namely
Choosing the appropriate transfer function h (U, v) is significant for low-pass filtering in the frequency domain. The commonly used transfer functions are ladder function, exponential function, Butterworth function and so on.
Several low-pass filters commonly used in frequency domain are ideal low-pass filters (Ideal circular iow-passfilter), Butterworth (Butterworth) Low-pass filters, exponential low-pass filters, and trapezoidal low-pass filters. These low-pass filters can play an important role in improving the noise-interfering components in the image.
There are two types of filtering methods used in the plane space domain of <2>:

One is the method of fitting the image, including n-order polynomial fitting, discrete orthogonal polynomial fitting, two-time surface fitting and other methods;

The other is the method of smoothing image, including domain average method, median filter method, gradient reciprocal weighting method, selective mask method and so on.

(ii) by frequency

<1> High-pass filtering: Edge extraction and enhancement. The gray-scale transformation of the edge region increases, that is, the frequency is high. Therefore, for high-pass filtering, the edge portion will be retained, and the non-edge portion will be filtered;

<2> Low-pass filtering: edges are smooth, and edge areas are smoothly transitioned.

Attached: Gaussian filter: Gaussian filter is a linear smoothing filter, that is, low-pass filtering, suitable for eliminating Gaussian noise, widely used in image processing of noise reduction process. In layman's words, Gaussian filtering is the process of weighted averaging of an entire image, and the value of each pixel is obtained by weighting the value of the other pixels in itself and in the neighborhood. Gaussian smoothing filters are very effective in suppressing noise that obeys a normal distribution. the 3x3 mask is as follows:

X1/16

As you can see from the structured mask, the location in the center of the mask is larger than any other pixel, so the pixel given in the mean calculation is more important. Pixels farther away from the mask center are less important to reduce blur in smoothing.

(c) Linear and nonlinear filtering

<1> Linear Filter: The original data and filter result of linear filter is an arithmetic operation, i.e., such as (1) mean filter (average of pixel gray value in template), (2) Gaussian filter (Gaussian weighted average value), etc. Since the linear filter is an arithmetic operation with a fixed template, the transfer function of the filter is deterministic and unique (the transfer function is the Fourier transform of the template).

         <2> Nonlinear Filter: The original data and filter result of nonlinear filter is a kind of logical relation, that is realized by logical operation, such as maximum filter, minimum value filter, median filter, etc., is realized by comparing the size of gray value in certain neighborhood, there is no fixed template, Thus there is no specific transfer function (because there is no template for the Fourier transform), and the expansion and corrosion are also achieved by the maximum and minimum filters. Five kinds of common nonlinear filter operators, these five kinds of filter operators have different effects on different images, the most commonly used is median filter, because it has the best effect and the least information loss. 

(1). Max-Value filtering

The maximal value filter is to select the maximum value of the pixel field as the pixel value of the pity Dorado, and the noise with lower gray value is reduced, and also can be used as the expansion operation in the morphology.

Maximum filtering can be expressed as: Maximum (A) =max[a (X+I,Y+J)] (x, y) is M

Note: (X+I,Y+J) is the coordinate defined on the image, (I,J) is the coordinate defined on the template m. M is the template for the operation.

(2). Minimum filtering (as opposed to maxima filtering)

(3). Midpoint filtering

Midpoint filtering is often used to remove short-tailed noises in images, such as Gaussian noise and uniformly distributed noise. The average value of the maximum and minimum values of the grayscale in a given window at the output of the end filter;

Midpoint (A) = (Max[a (x+i,y+j)]+min[a (X+I,Y+J)])/2 (x, y) belongs to M

Note: (X+I,Y+J) is the coordinate defined on the image, (I,J) is the coordinate defined on the template m. M is the template for the operation.

(4). Median filter

Median filtering eliminates long-tailed noise in images, such as negative exponential noise and salt and pepper noise. When the noise is eliminated, the median filter is very fuzzy to the image noise (influenced by the template size), and the median filter essentially replaces the grayscale of the template center pixel with the median value of the pixel grayscale in the template. Median filter has better performance than mean filter in eliminating salt and pepper noise and preserving spatial details of images.

Median (A) =median[a (X+I,Y+J)] (x, y) belongs to M

Note: (X+I,Y+J) is the coordinate defined on the image, (I,J) is the coordinate defined on the template m. M is the template for the operation.

(5). Weighted median filter (improvement of median filter)

The weighted median filter is improved on the basis of median filtering, and its performance is better than median filter in some degree.

Here's how it's improved on the algorithm: by example

If the size of the selected template is 5, then the template is uniquely defined as:

1 1 5) 1 1

1 1 5) 1 1

5 5 5) 5 5

1 1 5) 1 1

1 1 5) 1 1

The number of pity Dorado pixels in a sequence that appear in the series. The image is then smoothed according to the template. The experiment proves that the method is better than the traditional median filter. Of course there are other ways;

Attached: Noise          (Gaussian noise: Refers to the noise is subjected to Gaussian distribution, that is, a certain intensity of the number of noise points, the farther away from the intensity of the number of noise points, and this rule obeys the Gaussian distribution. Gaussian noise is an additive noise, that is, the noise is added directly to the original image, so it can be filtered with a linear filter.              Salt and pepper noise: similar to the salt and pepper in the image, so named, is a lot of white spots or black spots on the image of the noise, such as the TV snowflakes noise. Salt and pepper noise can be considered as a kind of logic noise, the result of filtering by linear filter is bad, and the median filter filter can be used to get better results. ）

3. Image Smoothing

Image smoothing Images Smoothing: suppressing, weakening, or eliminating details, mutations, edges, and noises in images is the smoothing of images. Image smoothing is a low-pass filter for images, which can be implemented in spatial domain or frequency domain. The spatial domain image smoothing method mainly uses the low-pass convolution filter, the median filter and so on; Frequency domain image smoothing commonly used low-pass filter has low-pass trapezoid filter, low-pass Gaussian filter, low-pass index filter, Butterworth low-pass filter, etc.

Relationship:

Image convolution: a means of implementation, whether it is filtering or anything else, it can be said that mathematics in the image processing an extension.

Image filtering: An image processing method to achieve different purposes.

Image smoothing: It's actually low-pass filtering.

Basic concepts of image processing-convolution, filtering, smoothing (reprint)