Image Fusion (Iii.)--Laplace Pyramid

2. Laplace Pyramid Fusion

Image Pyramid Method principle is: to participate in the fusion of each image decomposition into a multi-scale pyramid image sequence, the low-resolution image in the upper layer, high-resolution image in the lower layer, the upper image size of the previous layer of image size of 1/4. The number of layers is 0,1,2 ... N. The pyramid of all images is fused with certain rules on the corresponding layer, then the synthetic pyramid can be obtained and then reconstructed by the inverse process generated by the pyramid to obtain the Fusion pyramid. This general idea is all based on the pyramid fusion of the algorithm process, the difference is that the decomposition of the structure of the pyramid, each layer of the fusion rules are different, the method of reconstruction. The Pyramid method first realizes this idea, then the Wavelet method further consummates and develops this kind of multi-scale integration idea.

2.1, the principle of elaboration(1) Gauss pyramid

The Gaussian pyramid is the most basic image tower. First, the original image as the lowest image G0 (the No. 0 layer of the Gaussian pyramid), using the Gaussian kernel (5*5) to convolution, and then the convolution of the image after the next sample (remove even rows and columns) to obtain a previous layer of image G1, the image as input, repeat convolution and down sampling operation to get a higher layer of image, Iterate over multiple times to form a pyramid-shaped image data structure, the Gaussian pyramid.

The Gaussian pyramid is constructed as follows: Assume that the L-layer image of the Gaussian pyramid is GL:

In the formula, n is the top layer of the Gaussian pyramid, and the number of rows and columns W (m,n) of the Gaussian pyramid, respectively, of RL and Cl is a two-dimensional separable 5*5 window function, the expression is:

　　By G0,G1,,,GN, a Gaussian pyramid is formed, where G0 is the bottom of the Gaussian pyramid (the same as the original image) GN is the top of the pyramid. This shows that the current layer of Gaussian pyramid image is the first layer of the image is Gaussian low-pass filtering, and then interlaced and separated by 2 of the sample generated. The previous layer of image size is 4 times times the size of the current layer image in turn.

Gaussian pyramids can be obtained using the Pyrdown function in OpenCV.

(2) Laplace pyramid

During the operation of Gauss Pyramid, some high-frequency details are lost when the image is convolution and under-sampling. To describe these high-frequency information, people define the Laplace pyramid (Laplacian Pyramid, LP). With each layer of Gaussian pyramid, minus the predicted image after sampling and Gaussian convolution on the previous image, a series of difference images are given as LP decomposition images.

The Gl interpolation method is *GL to enlarge the image so that the size of the *GL is the same as the size of the *gl-1, that is, the amplification operator expand

Expressed as:

The formula implements two steps: inserting 0 in even rows and columns, and then filtering with the Gaussian kernel in the lower sample, resulting in an image as large as the L-1 layer.

Other:

n is the layer number of the top layer of the Laplace Pyramid LPL is the first L-layer image of the Laplace pyramid decomposition. by LP0,LP1, LP2 ... The pyramid formed by LPN is the Laplace pyramid. Each layer of its L0 image is the Gaussian pyramid of the G0 image and its layer image G1 the image *g1 after interpolation amplification, this process is equivalent to bandpass filtering, so the Laplace pyramid is also known as the band-pass pyramid decomposition.

Interpolation method: There is a function Pyrup implemented in OPENCV. Can get *g1. Then the Laplace pyramid can be obtained by subtracting the two functions.

The Laplace pyramid of each image needs to be fused to the corresponding level of the image, the specific fusion rules have, take big, take small, and so on.

(3) refactoring

For the fused Laplace pyramid, the corresponding Gaussian pyramid can be restored, and finally the original image is G0, starting from its top layer and pushing the bottom-by-layer. is to start using the interpolation method from the top level.

2.2. Fusion Application

The purpose of the image Laplace pyramid decomposition is to separate the source image into different spatial frequency bands, and the fusion process is carried out separately on each spatial frequencies layer, so that different fusion operators can be used to highlight the characteristics and details of the specific frequency bands in the different frequency bands of different decomposition layers. That is, it is possible to combine features and details from different images.

(1) Top-level processing

The Lal and LBL are the first L-layer images of the source image, A and b after the Laplace pyramid decomposition, and the result of fusion is LFL. When L=n, the LAN and LBN are the top-level images, respectively, of the source image, A and B, after the Laplace pyramid decomposition. For the fusion of top-level images, first calculate the area average of m*n (m, n Odd and M >= 3, n >= 3) with its various pixel-centric regions:

where IX and iy respectively are pixel f (x, y) in the X and Y direction of the first-order difference, defined as follows:

Therefore, for each pixel in the top-level image LAN (I, J) and LBn (i, J) can get the corresponding region average gradient ga (i, J) and GB (I, J). The average gradient reflects the subtle nuances and texture variations in the image, as well as the sharpness of the image. In general, the larger the average gradient and the richness of the image level, the clearer the image. Therefore, the fusion result of the top-level image is:

(2) processing at all levels

When 0<l<n, the region energy is computed first for the image of the L-layer decomposed by the Laplace pyramid:

Then the fusion result of the other layers image is:

After the fusion image LF1, LF2 and LFN are obtained at all levels of the pyramid. The final fusion image can be obtained through the previous refactoring.

The second type of fusion rule:

The fusion strategy of using the highest level coefficient to take the average and the absolute value of the remaining layer coefficients is large. The image's coefficient (gray value) is closer to the gray value of the clearer image, which shows that the fusion effect is good.

Image Fusion (Iii.)--Laplace Pyramid