Tssao Temporal screen-space Ambient occlusion (Unity3d 5 example implementation)

Prerequisites

Ambient light (ambient occlusion) is a kind of GI, its simplified form Ssao can be described by "micro-efficient", consumption is very small, the results are very good.
The essence of ambient occlusion (ambient occlusion) is to calculate how many directions are blocked (such as 1.2.1) within the range of the hemispherical surface of a point, and then adjust the surface color according to it. If real-time rendering is consumed, it is generally used in the game (Screen-space Ambient occlusion) SSAO. Ssao uses depth buffer to approximate a discrete scene, such as 1.2.2, to obtain a fixed overhead and is relatively straightforward to implement. However, due to the small number of Ssao sampling, so the result is not ideal, it is easy to produce noise. Noise is more pronounced when the object moves. And Ssao is low-frequency low-resolution. So we use the temporal coherence (temporal coherence) Ssao to solve this problem.

Reverse two-time projection technology (reverse reprojection) lets us reuse the pixels of the previous frame and refine them over time. This keeps the number of sample points per frame low, and adds up to hundreds of sample points in a short period of time. So temporal coherence is well suited to AO. And independent of the light source and viewpoint, AO is related only to the geometric structure of the local neighborhood (local neighborhood) and Ssao to the scene structure of the Pixel neighborhood. This article mainly explains the use of two-time projection technology (reverse reprojection) to improve the quality of Ssao in a deferred rendering scenario.

AO (Ambient occlusion)

From a physical point of view, the AO is a diffuse reflection of sky light. The calculation formula is as follows:

W is all directions on the hemisphere, V is the binary visibility function (binary visibility functions), and if it is visible V (p,w) = 1, it is 0 if it is blocked. D is a monotonically decreasing function between 0-1, which decreases as the distance from p to ξ increases and the ξ is the intersection of the Rays to the nearest surface. To simplify, D is a step function. Although smooth attenuation has provided good results, the sampling radius is still the biggest problem

SSAO (screen Space Ambient occlusion)

Ssao is an approximate AO method, which is done in screen space.
We can determine the blocking in two ways:
1. The distance between the sampling point and the current depth
2. Distance between the sample point and the current point

Can be expressed in the following formula:

Si is the sampling point, p is the current point and C is contribution
To approximate equation 1.1 using the Monte Carlo integration method, the contribution function is as follows:

The direction Ray of AO is replaced by the sampling point around p, so here the V (p,si) binary visibility function (binary visibility function) is visible at p point if the sample point Si, if visible is 0, not visible is 1. V is relative to the current z-buffer. Some Ssao methods omit depth test, and their SI contribution are based on the distance and incidence angle of p to SI. D is the attenuation of the p to Si distance.

reverse two-time projection (Reverse reprojection)

Reverse two-time projection combines the current frame pixel with pixels from the same world space position as the previous frame, and can reuse the previous frame's pixels at the current frame. This method can be used in many techniques, such as shadow Mapping, antialiasing, motion blur, and so on. We present the AO information from the previous frame to the render target. In a static geometry, a two-time projection is a constant throughout the frame. Using the viewpoint (V) and projection matrix (P) of the previous frame (f-1) and the current frame (f), T is the post-perspective position of the pixel

We use this frame of deferred render storage and the previous frame eye-linear depth to find world space coordinates, or you can store the world coordinates directly into a render target (the blogger thinks the latter is more convenient and eliminates the repetition calculation). We only need to convert the current world coordinate Pf to the previous frame of the view-projection matrix pf-1vf-1 to get tf-1
We calculate the find texture coordinate texf-1 based on tf-1 to find the AO buffer of the previous frame. Apply perspective Division and Scale results within 0-1 range.

Dynamic Objects

If the equation 1.4 is not valid in a dynamic scenario, the two-time projection is based on the transformation of the dynamic object. The solution is to perform two vertex transformations, one at a time using the current frame conversion parameters (modeling matrix, skinning.), and the other using the parameters from the previous frame. We couldn't store the conversion parameters, so we saved the PF-1-PF to a render target,

This offset can be stored here using a render target with a lower accuracy than depth, which can be used with 16bit.

Simply explain the unity render target format

ARGB32	Color Rendering Map format , per channel 8 bits
Depth	a Depth Stickers
Argbhalf	Color Rendering Map format , per channel - bit
Shadowmap	a local system. Shadowmap Render map Format
RGB565	Color Rendering Map Format
ARGB4444	Color Rendering Map format , per channel 4 bit
ARGB1555	Color Rendering Map format , RGB Channel 5 bit , Alpha Channel 1 bit
Default	default color rendering map format, Frame Buffer also this format
ARGB2101010	color Rendering map format. Color Ten bits,Alpha 2 bits
Defaulthdr	default HDR Color Rendering map format, HDR of the Frame Buffer also this format
Argbfloat	Color Rendering Map format , per channel + bit floating point value
Rgfloat	two different colors (RG) Render map Format , per channel + bit floating point value
Rghalf	two different colors (RG) Render map Format , per channel - bit floating point value
Rfloat	scalar (R) render map format, + bit floating point value
Rhalf	scalar (R) render map format, - bit floating point value
R8	scalar (R) render map format, 8 bit fixed point.
Argbint	Four channels (ARGB) Render map Format , per channel + bit signed integer
Rgint	two channels (RG) Render map Format , per channel + bit signed integer
RInt	scalar (R) render map format, + bit signed integer

refined Ssao over time

At the current frame F, we calculate the number of new contribution cf,k as valid (visual V>0) sample points

The JF (p) is the number of all used sample points.
Combining the information from the previous frame we get the AO method.

The weight WF (p) is the cumulative number of all sample points. There is a maximum value as a threshold value.
Theoretically, this method can be arbitrary sampling, but in practice, it is not possible, because two times the projection is not accurate, and the reconstruction requires bilateral filtering, every time, two times each projection makes this error intensified, this error brings obvious ambiguity, and will be more and more blurred with time. Furthermore, the new sampling point gradually tends to zero, and the previous sampling will never be eliminated. So we use the previously defined Wmax threshold to clamp WF, making the old contributions fade over time. Here the Wmax tends to be infinity.

The blogger puts the W value of each frame in the Y channel, and the final AO value exists in the X channel, and the depth exists in the Z channel.

Starting index JF exists in channel a
Gets the index value from the center of the Pixel, and the center of the pixel is calculated based on the texture coordinates obtained above texf-1,res x, Y is the current frame buffer resolution.

detection and processing of invalid pixels

When we re-projecting a fragment, we need to determine whether the previous frame corresponds to the current pixel, which means that the AO of the previous frame we stored is valid. We use the following three conditions to determine whether it is valid: 1. The sampling neighborhood of the current fragment nonblocking 2.fragment is changed before the outside of the frame 3.fragment.

No blocking is judged

We use the previous frame and the current frame depth to determine whether there is no blocking, the current frame depth value is DF, the previous frame is df-1.

The upper formula can produce a stable result in a large scene, has a wide depth range, is insensitive to the near surface, and is very sensitive to the long distance. In this case, we discard the previous frame, set the wf-1 to 0, and then calculate the new AO.

determine the variation of the sampling neighborhood

Judging the non-blocking step has avoided most of the time coherence flaws, but is only confined to stationary scenes. However, Ssao collects information from neighboring pixels by using space sampling kernel. So, in a dynamic scenario we need to consider that the dynamic moving object of the current pixel-point neighborhood will affect the current pixel, even in non-blocking pixels.
We use sampling in AO to do two things one is to calculate the current contribution Cf (p) and the other is to detect the validity. Detection principle such as
The effective sampling point SI and pixel p are estimated by sampling and changing the relative position of the pixel:

The sif-1 is calculated from the previously stored offset vector (offset vectors) and Si (the method mentioned above). Use only the samples in front of the tangent face.

Smooth Failure Place

In the above method we use a binary threshold to detect non-blocking, and we check all samples to discard the previous frame value. However, on a slowly deformed surface, the AO change is also very slow, at which point we do not need to completely discard off the value of a frame, using a new method to find the degree of change, the following formula:
Calculates a confidence in the range of 0-1 to indicate how effective the previous Ssao is.

The smoothness of the S control is not valid in the range of 15-30, as shown in the distribution. If the relevant distance does not change, δ (x) = 0, as δ (SI) increases confidence to zero.

All the confidence in the previous AO:

Multiply it with the previous weight weight (equation 1.6), in order to prevent flicker, convergence less than a certain threshold when we do not reuse the same sample, such as Conv (p) <0.5 (equation 1.7).
The high S will remove the defects of temporal coherence, but will produce many noise.

Handling Frame-buffer Boundaries

The information at the frame boundary is not correct, so we detect the sample at the boundary at the previous frame, without using the above smoothing failure at the boundary, discard it directly, and not sampling at the boundary when the current frame is calculated confidence.

Adaptive convergence-aware Spatial filtering.

Ssao usually uses a spatial filter to mitigate the noise caused by insufficient sampling rates. Tssao also uses a spatial filter, through the sampling point and the current point in the world space distance, this adaptive spatial filtering can automatically take into account the depth gap, and detect the depth gap is very large discontinuous places:

X is the sampling point, K (p) is a normalization factor, and G is a spatial filter kernel (such as Gaussian filtering).

Give the filter code

<span style= "FONT-SIZE:14PX;" >for (int i = 0; i < 3; i++) {for (int j = 0; J < 3; J + +) {Uv_sam = UV + FLOAT2 ( -1 + i,-1 + j)/_size;buffer = Te X2d (_aotex, uv_sam); conv = Buffer.y;c = Buffer.x;pos = GetPosition (uv,0);p Os_sam = GetPosition (uv_sam,0); g = g (Distance (p Os,pos_sam)); Kp + = G*conv; Ao + = Kp*c;}} </span>

Sampling Method

The sampling distance should not be too close, judging the blockage is very few, in the radius of the hemisphere or half of the radius of such a distance between the best.
You can refer to the previously written article Super Sampling supersampling method Summary, here recommended Halton sequence methods

Halton sequence

3D Halton Sequence

The X,z component calculates the unit sphere in the random direction, and the Y component sets the sampling radius R.
In game programming GEMS8 uses a semi-random method semi-random 3D Vectors
Random direction vectors and distances around normals
Here's the code:

FLOAT3 reflect (Float3 vsample, Float3 vnormal) {return normalize (vsample–2.0f * dot (vsample, vnormal) * vnormal);} float3x3 makerotation (float fangle, float3 vaxis) {float fs;float fc;sincos (fangle, FS, FC); float FXX       = vaxis.x * Vaxis.x;float Fyy       = vaxis.y * vaxis.y;float FZZ       = vaxis.z * vaxis.z;float FXY       = vaxis.x * Vaxis.y;float fYZ
   = vaxis.y * vaxis.z;float FZX       = vaxis.z * vaxis.x;float FXS       = vaxis.x * fs;float FYS       = vaxis.y * Fs;float FZ S       = vaxis.z * fs;float Fonec      = 1.0f-fc;float3x3 result = float3x3 (Fonec * FXX +  FC, Fonec * FXY + FZS, fOn EC * Fzx-fys,fonec * Fxy-fzs, Fonec * fyy +  FC, Fonec * fyz + Fxs,fonec * FZX + FYS, Fonec * FYZ-FXS, Fonec * f ZZ +  FC); return result;}

Float4 Postprocessssao (float3 i_vpos) {... const float c_scalingconstant = 256.0F;FLOAT3 Vrandomnormal = (Normalize (Tex (P_sssaonoise, VSCREENUV *  p_vsrcimagesize/c_scalingconstant). XYZ * 2.0f–1.0f)); float3x3 Rotmatrix = MakeRot ation (1.0f, vnormal); half faccumblock = 0.0f;for (int i = 0; i < Isamplecount; i++) {Float3 Vsamplepointdelta = ref Lect (P_vssaosamplepoints[i], vrandomnormal); float Fblock = Testocclusion (Vviewpos, Vsamplepointdelta,p_ Focclusionradius,p_ffullocclusionthreshold, P_fnoocclusionthreshold, P_focclusionpower)) {fAccumBlock + = FBlock;} ...}

Results

The blogger achieves a total of three steps:

1.SSAO

2.TSSAO

3.Filter

Give a three-step implementation result

For Ssao with no filtering

For Tssao with no filtering

For a filtered Tssao

For diffuse Color no lighting and shading

For diffuse Color and filtered Tssao

For diffuse color and filtered Tssao and illumination colors

Original

Original and Filtered Tssao

Summary

For the Ssao itself there is a problem is that the object is near or too far away from the bad, because in the fixed sampling range, too far object sampling range is too large to ignore the distance of the details of the blockage, near and due to the small sampling range, resulting in almost the same position, almost no blocking, The method to solve this problem is to adjust the sampling range dynamically according to the distance, which is a kind of optimization processing of Ssao.
Ssao itself is a low-resolution processing, because the performance of sampling points too little, sampling rate is too small, but Tssao effectively solves this problem, so that the sampling point with time, and can handle dynamic scenes well, the effect is much better than Ssao, and more correct, noise is very few.

By the end of this article,

by the way, the internship work, haha

Recent results: Recently made some rendering

Email: [Email protected]

------by wolf96

Tssao Temporal screen-space Ambient occlusion (Unity3d 5 example implementation)