Original link: http://blog.csdn.net/bugrunner/article/details/7170471
Another similar English material: Http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MANDUCHI1/Bilateral_Filtering.html#Index
1. Introduction
Image smoothing is an important operation, and there are many mature algorithms. Here is a brief introduction of the bilateral method (bilateral filtering), which is mainly due to the previous period of time to do Ssao, need to use bilateral blur algorithm for noise reduction. Bilateral blur is an important feature relative to the traditional Gaussian blur that can be kept edge (edge perseving), a feature that is useful for some blurred images. The typical Gaussian Blur takes into account the spatial distance between pixels, but does not take into account the similarity between pixel values, so the fuzzy result we get is usually a blur of the whole picture. The improvement of bilateral blur is that it not only takes into account the relationship between pixels in space distance, but also considers the similarity between pixels, so it can keep the rough block of the original image and keep the edge. In the game engine's post blur algorithm, bilateral blur is often used, such as Ssao noise reduction.
2. Principle
In the filtering algorithm, the pixel value on the target point is usually determined by the value of a small local neighbor pixel in the vicinity of its location. The realization of 2D Gaussian filter is to assign different Gaussian weights to the surrounding pixel values, and get the final result of the current point after weighted average. The Gaussian weighting factor here is generated using a spatial distance between two pixels (2D in the image). By the Gaussian distribution curve, it is found that the closer point of the target pixel contributes to the final result, the smaller the inverse. The description of its formulation is generally described as follows:
C is the Gaussian weight based on the spatial distance, which is used to unit the result.
Gaussian filtering has a good performance in the low-pass filter algorithm, but it has another problem, that is, only consider the spatial position of the relationship between pixels, so the result of filtering will lose the edge of information. The edge here mainly refers to the image of the main different color areas (such as blue sky, black hair, etc.), and bilateral is in the Gaussian Blur added another weight division to solve this problem. The retention of edges in bilateral filtering is achieved by the following expression:
s is a Gaussian weight based on the similarity between pixels and is also used to unit the results. The combination of the two can get the bilateral filter based on the spatial distance and the similarity degree:
The unit division in the upper-form combines two Gaussian weights together, where the C and s calculations can be described in detail as follows:
and has
and has
The expressions given above are infinite integrals in space, and certainly not in pixelated images, and there is no need to do so, so they need to be discretized before they can be used. And there is no need for each local pixel from the entire image of the weighted operation, the distance beyond a certain degree of pixels actually affect the current target pixel is very small, can be ignored. The discretization of a local sub-region can be reduced to the following form:
The above-mentioned theoretical formulae constitute the basis of bilateral filter implementation. In order to intuitively understand the difference between Gaussian filtering and bilateral filtering, we can see the basis from the following illustrations. Assuming that the target source image is a noisy image (generated automatically by the program) in the following area, the center of the blue Box is the location of the target pixel, then the corresponding Gaussian weight and the bilateral weight factor 3D visualize the shape of the current pixel as shown in the following two figure:
The picture on the left is the original noise image, the weights for Gaussian sampling in the middle, and the weights for bilateral samples on the right. It can be seen that bilateral added a similar degree of division to the left side of the source image with the current pixel difference is too large to filter the point, so that the edge is good to maintain. In order to visually observe the difference between the two, use Matlab to draw the graph in the height of the map in the following ways:
The above three graphs are from left to right: bilateral filtering, original image, Gaussian filtering. It is obvious from the height map that the difference between the two methods of bilateral and Gaussian, the former better maintains the gradient at the edge, and in Gaussian filtering, because the variation at the edge is linear, the gradient is used to show the state of the gradients, This is shown in the image as the boundary is lost (an example of the image can be seen in the following).
3. Code implementation
With the above theory after the implementation of bilateral filter is relatively simple, in fact, it also with the ordinary Gaussian blur not much difference. This includes 3 parts: Weight factor generation based on spatial distance, generation of weight factor based on similarity, and calculation of final filter color.
3.1 Spatial Weight
This is the usual method of calculating the Gaussian weights used in Gaussian blur, which is calculated mainly by the distance between two pixel and using the following formula:
It represents the distance between two pixels, such as the distance between the current pixel and a pixel immediately to its right, which we can calculate, i.e. two two-dimensional vectors {0, 0} , and {0, 1 the Euclidean distance between the}. Gaussian blur can be done by directly calculating the Gaussian weights on an area and then the unit.
3.2 Similarity Weight
Similar to a distance-based Gaussian weight calculation, except that it is no longer based on the spatial distance between the two pixel, but on its similarity (or the distance between two pixel values).
Where the distance between the two pixel values is expressed, you can directly use the difference between their grayscale values or the Euclidean distance between the RGB vectors.
3.3 Color Filtering
With the weighting factors necessary for the above two parts, the implementation of the concrete bilateral filter is the same as the normal Gaussian filter. The main part of the code is as follows:
UCHAR3 Bbcolor (intPosX,intPosY) { intCenteritemindex = PosY * PicWidth4 + PosX *3, Neighbouritemindex; intWeightindex; DoubleGsaccumweight =0; DoubleAccumcolor =0; //calculates the Gaussian weights at each sample point, including the Closeness,similarity for(inti =-number; I <= number; ++i) { for(intj =-number; J <= number; ++j) {Weightindex= (i + number) * (number *2+1) + (J +Number ); Neighbouritemindex= Min (Noiseimageheight-1, Max (0, PosY + J * Radius) * PicWidth4 +min (noiseimagewidth-1, Max (0, PosX + i * radius)) *3; Pcsweight[weightindex]=lookupgsweighttable (Psrcdatabuffer[neighbouritemindex], psrcdatabuffer[centeritemindex]); Pcsweight[weightindex]= Pgsweight[weightindex] *Pgcweight[weightindex]; Gsaccumweight+=Pcsweight[weightindex]; } } //The weighted factor of the unitGsaccumweight =1/Gsaccumweight; for(inti =-number; I <= number; ++i) { for(intj =-number; J <= number; ++j) {Weightindex= (i + number) * (number *2+1) + (J +Number ); Pcsweight[weightindex]*=Gsaccumweight; } } //calculates the final color and returns for(inti =-number; I <= number; ++i) { for(intj =-number; J <= number; ++j) {Weightindex= (i + number) * (number *2+1) + (J +Number ); Neighbouritemindex= Min (Noiseimageheight-1, Max (0, PosY + J * Radius) * PicWidth4 +min (noiseimagewidth-1, Max (0, PosX + i * radius)) *3; Accumcolor+ = Psrcdatabuffer[neighbouritemindex +0] *Pcsweight[weightindex]; } } returnUCHAR3 (Accumcolor, Accumcolor, Accumcolor); }
The weight of the division of Similarity is mainly based on the color difference between the two pixel to calculate the surface. For grayscale graphs, the range of this difference is predictable, that is, [-255, 255], so in order to improve the efficiency of the calculation we can calculate this part of the weight factor to generate a parallel table, in the use of fast query can be. The results of filtering several noisy images using the algorithm described above are as follows:
From left to right: bilateral filtering; original image; Gaussian filter. From the picture can be more clearly see the difference between the two algorithms, the most intuitive feeling difference is the use of Gaussian algorithm after the whole picture is a group of fuzzy State, while the bilateral filter can be better to maintain the original image of the region information, it seems that the mouth is still mouth, eye is the eye.
4. Use in Ssao
The edge decision function in the above implementation is mainly determined by the difference between the two pixel values, which is also a common way of perceiving common images. Of course, it is also possible to use other methods to determine the edges of other definitions, such as the depth of the scene or normal, depending on the demand situation. For example, the Ssao can be filtered directly using the depth value to row edge judgment. First, set a depth threshold to compare the depth difference between two points when making edge detection, and if the difference is greater than the threshold value, it is considered to be a different region, and this should be the boundary. The effect that you get with this method is shown in the following:
Gaussian filter
Bilateral filtering
After the Ssao image is filtered, a direct integration with the original image can result in the final rendering, as shown in the following:
Ssao off
Ssao Open
Bilateral Filtering (bilateral filtering) for SSAO (RPM)