Analysis of FFT Lens Effect

Reprinted please indicate the source for the klayge game engine, this article address for http://www.klayge.org/2012/03/22/fft%e9%95%9c%e5%a4%b4%e6%95%88%e6%9e%9c%e8%a7%a3%e6%9e%90/

 

The whitepaper of 3dmark11 highlights how to implement the Lens Effect Using FFT. Here, the lens effects include bloom and extensive light, which are generally done before HDR tone mapping.

Traditional Lens Effects

Bloom is the most common effect. Generally, a Gaussian Blur is used. However, such a result lacks a sense of attention, but it is only bright. In halo and other games, a complicated bloom method is used. It first downloads the HDR Image multiple times, and performs Gaussian Blur for each time, and then mixes it with a certain weight to obtain a rich bloom.

Bungie's bloom method, from lighting and material of Halo 3

 

Bloom itself is basically enough, but if you want to add more lens glare effects, such as the star effect of hdrlighting in dx sdk, you have to add more pass. In addition, if Blur's kernel is large, the speed will also be greatly reduced. For example, many games have begun to use horizontal lens glare similar to Star Trek 11, and almost require a full-screen kernel, which is hard to afford for real-time applications. (J.j. Abrams uses this horizontal lens flare in many films)

 

FFT Method

In fact, the effects of all these lenses are blur, that is, convolution of each pixel with a kernel. There is a convolution Theorem in mathematics:

That is

It indicates convolution, Fourier transformation, and inverse Fourier transformation. That is to say,The Fourier transformation of function convolution is the product of function Fourier transformation.. Therefore, the convolution required by the lens effects can be completed with a multiplication operation in the frequency domain after FFT. No matter how many layers of bloom, how many stars, and how many kernel you need,All are solved within 1 pass..

The entire algorithm framework is very simple:

1. Input Image FFT
2. Use the FFT kernel image and the result of 1 to multiply the number of pixels.
3. Inverse FFT

This completes all Lens Effects. For example:

Input Image (from the recently completed scene player)

 

Before FFT, downsample to 512 × 512 size is required, and only the bright part (bright pass) is retained ):

 

The result of FFT is the real part and the virtual part:

 

 

Kernel image, which is randomly drawn in paint.net

 

After the kernel also performs FFT, the two are multiplied by the plural,

 

 

 

 

 

Here, RE and Im represent the real and virtual parts respectively.

Result of inverse FFT

 

Overlay the original image

 

The only problem left is how to get the kernel image. Currently, I have come up with an interactive tool that is hand-drawn by art and displays the final result next to it. You can predefine some pattern and use them in the tool.

Future

The FFT lens effect algorithm has been fully explored. In the current experiment, the FFT calls the fftw library, and the next step is to implement the FFT (with the PS version and the CS version) on the GPU, so that the entire pipeline can be deployed on the GPU, replace the bloom section in the HDR post process.

FFT can also be used in other Convolution-related operations. In the future, I will explore the commonly used effects such as DOF, bokeh, motion blur, and ssao Using FFT. Again, still completed in a pass.