Learning OpenCV---(7) Linear Neighborhood Filtering session: Box filter, mean filter, Gaussian filter

in this article, we have discussed the more popular image filtering operation in OpenCV image processing technology. Image Filter Series article light ink ready to spend two times to update the time, this is the last chapter, for everyone to analyze the "box filter", "Mean Filter", "Gaussian filter" three kinds of common neighborhood filtering operations. As the "median filter" and "bilateral filter" of nonlinear filtering, we analyze it next time.

Because the article is very long, if the detailed chewing, may be indigestion. Here is a guideline for you, if you want to master the OPENCV linear filtering related three functions in this article: Boxfilter,blur and Gausslanblur of the use of the method, directly see the third part of the "shallow out" and the fourth "example" on the line.

In the later OpenCV series of articles, the light ink is prepared to each blog post knowledge points are divided into principles, in-depth, shallow out and examples of four parts to explain,

The first part is related to the image processing linear filtering theory, the second section of the "in-depth" part of the main in-depth OPENCV internal, leading everyone to appreciate the charm of OPENCV open source, OPENCV related source analysis, to opencv a deep understanding, to do a high-end atmosphere OPENCV users, The third part "shallow out" the main church everyone how to quickly get started the relevant OPENCVAPI functions described in the current article. And in the fourth part, light ink will prepare a comprehensive example of the detailed comments related to this article.

Give the structure of this article:

First, theory--related image processing concept introduction

second, in-depth--OPENCV source explanation

Three, shallow out--api function explanation

Iv. Examples--Detailed annotated blog post Companion program

I. Theory and concept explanation

<1> about smoothing

"Smoothing" (smoothing), also known as "obfuscation" (bluring), is a simple and highly used image processing method. There are many uses for smoothing, the most common being to reduce noise or distortion on the image . Smoothing is a very useful method when it comes to reducing image resolution.

<2> Image Filter and filter

First we look at the concept of a filter.

Image filtering: To suppress the noise of the target image under the condition that the image details are preserved, it is an indispensable operation in image processing, and its processing effect will directly affect the validity and reliability of subsequent image processing and analysis.

Eliminating the noise component in the image is called smoothing or filtering the image. The energy of a signal or an image is mostly concentrated in the low and middle frequencies of the amplitude spectrum and is very common, while at higher frequencies, the information of interest is often masked by noise. Therefore, a filter that reduces the amplitude of high-frequency components can reduce the effect of noise.

There are two purposes of image filtering: One is to extract the characteristics of the object as the feature pattern of image recognition, the other is to adapt to the requirements of image processing, eliminate the noise mixed with image digitization, and the requirements for image processing have two: (1) Can not damage the image contour and edge of important information ; (2) Make the image clear and visual effect good

Smoothing filtering is a low-frequency enhanced spatial filtering technique. There are two types of goals: one is fuzzy and the other is noise elimination.

The smoothing filtering in spatial domain is usually carried out by simple averaging method, which is to find the average luminance value of the neighboring cell points. The size of the neighborhood is directly related to the smoothing effect, the greater the smoothing effect of the neighborhood, but the larger the neighborhood, the greater the loss of the edge information, so that the output of the image becomes blurred, so it is necessary to select the neighborhood size reasonably.

On the filter, an image analogy is: we can think of the filter as a window containing a weighted coefficient, when using this filter to smooth image processing, the window is placed above the image, through this window to see the image we get.

There are many types of filters, in the new version of OpenCV, the following five commonly used image smoothing operations are provided, and they are encapsulated in a separate function, which is very convenient to use:  

• Box Filter--boxblur function
• Mean Filter (neighborhood average filter)--blur function
• Gaussian filter--gaussianblur function
• Median filter--medianblur function
• Bilateral filtering--bilateralfilter function

Today we are going to explain the box filter, mean filter and Gaussian filter as linear filter. Two kinds of nonlinear filtering operations-median filtering and bilateral filtering-we'll leave it to the next explanation.

<3> Introduction to Linear filters

linear filters: Often used to reject unwanted frequencies in the input signal or choose a desired frequency from many frequencies.

Several common linear filters are:

----Low -pass filters that allow low frequencies

----High-pass filters with higher frequencies allowed

----Band-pass filters that allow a range of frequencies to pass

----A band-stop filter that prevents a certain range of frequencies from passing and allows other frequencies to pass through

----allows all frequencies to pass, only to change the phase relation of the All-pass filter

----Block a narrow frequency range through the special band-stop filter, notch filter (band-stop filter)

<4> about filtering and blurring  

on the filter and blur, we tend to confuse the first contact, "a moment to say the filter, and then say the blur, what the heck ah"

It's okay, here we are to discern, to clear the obstacles for everyone.

As we have mentioned above, filtering is the operation that filters out specific band frequencies in a signal and is an important measure to suppress and prevent interference.

For illustrative purposes, take the Gaussian filter we often use as an example. We know that filtering can be divided into low-pass filter and high-pass filter two. Gaussian filter refers to the use of Gaussian function as a filter function of the filter operation, as to whether it is fuzzy, to see whether it is Gaussian low-pass or Gaussian high-pass, low-pass is fuzzy, high-pass is sharpening.

In fact, it is very simple, right:

Gaussian filtering refers to the filtering operation using Gaussian function as filter function.

Gaussian Blur is a Gaussian low-pass filter.

<5> neighborhood operator and linear neighborhood filtering

A neighborhood operator (local operator) is an operator that uses the pixel values around a given pixel to determine the final output value of this pixel. While the linear neighborhood filter is a common neighborhood operator, the output value of the pixel depends on the weighted sum of the input pixels and the specific process.

In addition to local tone adjustment, the neighborhood operator can be used for image filtering, smoothing and sharpening images, image edge enhancement or image noise removal. This article, we introduce the main character is the linear neighborhood filter operator, that is, with different weights to combine a small neighborhood of pixels, to get the desired processing effect.

Note: Neighborhood Filtering (convolution): The left image and the convolution of the intermediate image produce the right image. The blue-labeled pixels in the target image are calculated using the red-labeled pixels in the original image

the output pixel value of the linear filter processing is the weighted sum of the input pixel values:

The weighted sum is, we call it "kernel", the weighted coefficient of the filter, that is, "filter coefficient" of the filter.

The above formula can be simply written:

where F is the input pixel value, H is the weighted factor "kernel", and G indicates the output pixel value

In the new version of OpenCV, the following three commonly used linear filtering operations are provided, which are encapsulated in separate functions and are very convenient to use:

----Box Filter Boxblur function

----Mean-value filter blur function

----Gaussian filter Gaussianblur function

Let's take a brief introduction to them.

<6> Box Filter (box filter)

box filter is encapsulated in a function called Boxblur, where the Boxblur function uses a box filter to blur a picture, input from SRC, and output from DST.

The function prototypes are as follows:

1 void int ddepth, Size ksize, point Anchor=point (-1,-1), boolnormalize=trueint Boardtype=border_default);

Detailed parameters:

• The first parameter, the Inputarray type SRC, the input image, namely the source image, fills the Mat class the object to be able. This function is independent of the channel and can handle images of any number of channels, but it should be noted that the depth of the image to be processed should be one of cv_8u, cv_16u, Cv_16s, cv_32f, and cv_64f.

• The second parameter, the Outputarray type of DST, which is the target image, needs to have the same size and type as the source picture

• The third argument, the int type of the ddepth, the depth of the output image, 1 for the use of the original depth, i.e. src.depth ()

• The fourth parameter, size type (which is explained later in the size type), is the ksize of the kernel. Typically this is written in size (w,h) to represent the kernel (where w is the pixel width and h is the pixel height). Size (3,3) represents the size of the 3x3 kernel, and size (5,5) represents the size of the 5x5 kernel

• The fifth parameter, the point type of anchor, represents the anchor point (that is, the one that is smoothed), and note that he has the default value ( -1,-1). If the point coordinates are negative, then the center of the kernel is the anchor point, so the default value point ( -1,-1) indicates that the anchor is at the center of the nucleus.

• The sixth parameter, normalize of type bool, is the default value of True, an identifier that indicates whether the kernel is normalized by its region (normalized).

• The seventh parameter, the bordertype of type int, is used to infer some boundary pattern for the outer pixels of the image. There is a default value of Border_default, we generally do not care about it.

the Boxfilter () function box Filter uses the kernel as follows:

which

where F is the original, H is the core, G is the target graph, and when normalize=true, the box filter becomes our familiar mean filter. In other words, the mean filter is a special case after the Square filter normalization (normalized). Normalization, in which the amount to be processed is scaled to a range, such as (0,1), for uniform processing and visual quantification.

Instead of normalization (unnormalized), box filtering is used to calculate the integral properties within each pixel neighborhood, such as the covariance matrix of the inverse of the image used in the dense optical flow algorithm (dense optical flow algorithms) (covariance matrices of image derivatives)

If we want to calculate the sum of pixels in a variable window, you can use the integral () function.

<7> mean-value filter

Mean filter, is the simplest kind of filtering operation, each pixel of the output image is the average value of the pixels of the input image corresponding to the pixel in the kernel window (all pixel weighted coefficients are equal), in fact, it is the square filter after normalization.

In the following analysis of the source code, we will find that inside the blur function is actually called a bit of boxfilter.

Let's start with the content of the mean filter.

1) The theory analysis of mean value filter

Mean filter is a typical linear filtering algorithm, the main method is the neighborhood averaging method, that is, the average value of each pixel in an image region is substituted for the individual pixel values in the original image. It is generally necessary to give a template (kernel) to the target pixel on the image, which includes the neighboring pixels around it (for example, around 8 (3x3-1) pixels centered on the target pixel to form a filter template that removes the target pixel itself). The average of all the pixels in the template is used instead of the original pixel values. That is, the current pixel (x, y) of the processing, select a template, the template is composed of several pixels of its neighbors, to find the mean value of all the pixels in the template, and then assign the mean to the current pixel (x, y), as the image at that point in grayscale g (x, y), that is, a G (X , where m is the total number of pixels in the template that contain the current pixel.

     

2) Defect of mean filter

there is inherent flaw in the mean filter itself, that is, it can not protect the image details well, while the image denoising also destroys the detail part of the image, so that the image becomes blurred and the noise point can not be removed well.

3) Use mean filter--blur function in OpenCV

the role of the Blur function is to filter the input image src by means of the DST output

In the Blur function document, the kernel given is this:

This kernel is understood at a glance, is the mean value, that is, the blur function encapsulates the mean filter.

Prototype of the Blur function:

1 void blur (Inputarray src, outputarraydst, Size ksize, point Anchor=point (-1,-1int Bordertype=border_default);

• The first parameter, the Inputarray type SRC, the input image, namely the source image, fills the Mat class the object to be able. This function is independent of the channel and can handle images of any number of channels, but it should be noted that the depth of the image to be processed should be one of cv_8u, cv_16u, Cv_16s, cv_32f, and cv_64f.
• The second parameter, the Outputarray type of DST, which is the target image, needs to have the same size and type as the source picture. For example, you can use the Mat::clone, the source image as a template, to initialize the target image such as false replacement.
• The third parameter, the size type (which is explained later in the size type), is the ksize of the kernel. Typically this is written in size (w,h) to represent the kernel (where w is the pixel width and h is the pixel height). Size (3,3) represents the size of the 3x3 kernel, and size (5,5) represents the size of the 5x5 kernel
• The fourth parameter, the point type of anchor, represents the anchor point (that is, the one that is smoothed), and note that he has the default value ( -1,-1). If the point coordinates are negative, then the center of the kernel is the anchor point, so the default value point ( -1,-1) indicates that the anchor is at the center of the nucleus.
• The fifth parameter, the bordertype of type int, is used to infer some boundary pattern for the outer pixels of the image. There is a default value of Border_default, we generally do not care about it.

<8> Gaussian Filter

1) Theoretical analysis of Gaussian filter

Gaussian filtering is a linear smoothing filter, which is suitable for eliminating Gaussian noise and is widely used in the process of noise reduction in image processing. In layman's words, Gaussian filtering is the process of weighted averaging of an entire image, and the value of each pixel is obtained by weighted averaging of its own and other pixel values within the neighborhood. The specific operation of the Gaussian filter is to use a template (or convolution, mask) to scan each pixel in the image, using the weighted average gray value of the pixels in the neighborhood of the template to replace the value of the template center pixel.

We often say that Gaussian filtering is the most useful filtering operation, although it is used, the efficiency is often not the highest.

The image generated by the Gaussian blur technique is like a translucent screen looking at the image, which is significantly different from the bokeh of the lens and the shadows of ordinary illumination. Gaussian smoothing is also used in the pre-processing phase of computer vision algorithms to enhance image performance at different proportions (see scale space representation and scale space implementations). From the point of view of mathematics, the Gaussian blur process of image is the convolution of image and normal distribution. Since the normal distribution is also called Gaussian distribution, this technique is called Gaussian blur.

Blurring the image with the circular box will produce a more accurate out-of-focus imaging effect. Because the Fourier transform of the Gaussian function is another Gaussian function, Gaussian blur is a low-pass filtering operation for the image.

Gaussian filter is a kind of linear smoothing filter which chooses weights according to the shape of Gaussian function. Gaussian smoothing filters are very effective in suppressing noise that obeys a normal distribution. The one-dimensional 0 mean Gaussian function is:

The Gaussian distribution parameter sigma determines the width of the Gaussian function. For image processing, the two-dimensional 0 mean discrete Gaussian function is used as smoothing filter.

the Ivigos function is:

2) Using Gaussian filter--gaussianblur function in OpenCV

the role of the Gaussianblur function is to use a Gaussian filter to blur a picture, the input of the image SRC Gaussian filter after the use of DST output. It makes convolution operations on the source image and the specified Gaussian kernel function, and supports in-place filtering (in-placefiltering).

1  void Double Double sigmay=0

• The first parameter, the Inputarray type SRC, the input image, namely the source image, fills the Mat class the object to be able. It can be a single picture of any number of channels, but it should be noted that the image depth should be one of cv_8u,cv_16u, Cv_16s, cv_32f and cv_64f.
• The second parameter, the Outputarray type of DST, which is the target image, needs to have the same size and type as the source picture. For example, you can use the Mat::clone, the source image as a template, to initialize the target image such as false replacement.
• The third parameter, size of type ksize Gaussian kernel. where Ksize.width and ksize.height can be different, but they both have to be positive and odd. Alternatively, they can be zero, and they are all calculated by Sigma.
• The fourth parameter, a double of type Sigmax, represents the standard deviation of the Gaussian kernel function in the x direction.
• The fifth parameter, a double of type Sigmay, represents the standard deviation of the Gaussian kernel function in the y direction. If the Sigmay is zero, it is set to Sigmax, if both Sigmax and Sigmay are 0, then it is calculated by Ksize.width and ksize.height.
• For the sake of correctness of the result, it is best to assign the third parameter size, the fourth parameter Sigmax and the fifth parameter Sigmay all to.
• The sixth parameter, the bordertype of type int, is used to infer some boundary pattern for the outer pixels of the image. There is a default value of Border_default, we generally do not care about it.

Learning OpenCV---(7) Linear Neighborhood Filtering session: Box filter, mean filter, Gaussian filter