Introduction of three-time illumination rendering methods in game engine (take Unity3d as an example)

Important: The following three types of lighting implementations are mainly used in game engines that are common on the market:

Vertex illumination render Path detail Vertex Lit Rendering Path details

Forward render path Detail Forward Rendering path details

Details of deferred light rendering path Deferred Lighting Rendering path details

Take Unity3d as an example, the following will explain in detail the implementation of three kinds of lighting rendering methods, principles and defects.

vertex Illumination render path details Vertex Lit Rendering Path Details

Vertex Lit path generally renders each object in one pass, with lighting from all lights calculated at object vertices.

Vertex illumination render paths usually render objects in one channel, and all light sources are calculated on the vertex of the object.

It's the fastest rendering path and have widest hardware support (however, keep in Mind:it does don't work on consoles).

The vertex illumination render path is the fastest rendering path and has the widest range of hardware support (however, keep in mind that it does not work on a game console).

Since all lighting are calculated at vertex level, this rendering path does isn't support for most of the Per-pixel effects:shadows, Normal mapping, light cookies, highly detailed specular highlights is not supported.

Since all illumination is computed at the vertex level, this render path does not support most pixel-by-pixels rendering effects: shadows, normal maps, light masks, high-precision highlights, for example.

Forward render path Detail Forward Rendering path details

Forward Rendering path renders each object in one or more passes, depending on lights that affect the object. Lights themselves is also treated differently by Forward Rendering, depending on their settings and intensity.

Depending on the light source that affects the object, the forward rendering path renders the object with a single or multiple channels. In forward rendering, the light source itself will be treated differently depending on their settings and intensity.

Implementation Details Implementation Detail

In Forward Rendering, some number of brightest lights this affect each object is rendered in fully per-pixel lit mode. Then, the up to 4 point lights is calculated Per-vertex. The other lights was computed as spherical harmonics (SH), which is much faster but was only a approximation. Whether a light would be per-pixel light or not are dependent on this:

In forward rendering, the lightest light sources that affect an object use pixel-by-light mode. Next, up to 4 point lights are calculated as vertex-by-pixel rendering. Other light sources will be calculated in the form of spherical harmonic (spherical harmonics), and the spherical harmonic technique can be calculated quickly but only with approximate values. Determine whether a light source is a per-pixel light source according to the following rules:

• Lights that has their Render Mode set to not Important is always Per-vertex or SH. A light source that is set to be unimportant (not Important) is calculated in the form of a vertex or spherical harmony
• Brightest directional light are always per-pixel. Brightest directional light is a pixel light source
• Lights that has their Render Mode set to Important is always per-pixel. Render mode is set important (Important) light source is the pixel light source
• If the above results in less lights than current Pixel light Count quality Setting and then more lights  is rendered per-pixel, in order of decreasing brightness. If the number of pixel light sources based on the above rules is less than the number of pixel lights in the quality setting (Pixel light count), more light sources will be rendered in pixel-wise to reduce brightness

Rendering of each object happens as follows:

Render each object in the following ways:

• Base Pass applies one per-pixel directional light and all per-vertex/sh lights. The underlying channel renders a pixel-by-point light and all the vertex/sphere-by-layer harmonic light.
• Other Per-pixel lights is rendered in additional passes, one pass for each light. Other pixel-by-image light are rendered in the attached channel, requiring one channel for each light source

For example, if there are some object that's affected by a number of lights (a circle in a picture below, affected by light s A to H):

For example, if an object is affected by several light sources (the circle in which the light is affected by A to h)

Let's assume lights A to H has the same color & intensity, all of the them has Auto rendering mode, so they would is Sorted in exactly the order for this object. The brightest lights would be rendered in Per-pixel lit mode (A to D), then up to 4 lights in Per-vertex lit mode (D to G), And finally the rest of lights in SH (G to H):

Assuming that light sources a to H have the same color and intensity, and that their rendering modes are automatic (auto), they are strictly sorted by their name. The brightest light source is rendered in pixel-per-light mode (A through D), and then up to 4 lights are rendered in per-vertex illumination mode (d to G), and other light sources are rendered in spherical tones (g to h).

Note that light groups overlap; For example last Per-pixel light blends to Per-vertex lit mode so there is less "light popping" as objects and lights M Ove around.

Note that there is overlap between the different light groups, such as the last pixel-by-point light that is rendered in the light-per-vertex mode, which reduces the "light jump" that can occur when objects and lights move.

Base Pass Basic Channel

Base pass renders object with one per-pixel directional light and all SH lights. This pass also adds any lightmaps, ambient and emissive lighting from the shader. Directional light rendered on this pass can has Shadows. Note that lightmapped objects does not get the illumination from SH lights.

The underlying channel renders the object with a pixel-by-direction light and all spherical harmonic light. This channel is also responsible for rendering the illumination map, ambient light, and spontaneous light in the shader. The direction light rendered in this channel can produce shadows. It is important to note that objects that use a light map do not get light from the spherical and light.

Additional Passes Additional Channels

Additional passes is rendered for each Additional per-pixel light that affect this object. Lights in these passes can ' t has shadows (so in result, Forward Rendering supports one directional light with shadows).

Additional channels are used to render other pixel-wise light sources that affect objects. The lights rendered in these channels cannot produce shadows (so forward rendering supports a directional light that can produce shadows).

Performance Considerations Performance Considerations

Spherical harmonics Lights is very fast to render. They has a tiny cost on the CPU, and was actually free for the GPU to apply (that's, base pass always computes SH Lighting; But due to the the-the-lights work, the cost is exactly the same no matter how many SH lights is there).

Render spherical and light quickly. They spend very little CPU time and actually do not have to spend any GPU computing time (in other words, the underlying channel calculates the spherical harmonic illumination, but because the spherical harmonic light is computed, the time spent calculating them is the same regardless of the number of spherical harmonic lights).

The downsides of SH lights are:

The disadvantages of spherical harmonic light sources are:

• They is computed at object ' s vertices, not pixels.  This means they don't support light Cookies or normal maps. They calculate the vertex of the object rather than the pixel. This means that they do not support shadow masks and discovery maps.
• SH lighting is very low frequency. You can ' t has sharp lighting transitions with SH lights.  They is also only affecting the diffuse lighting (too low frequency for specular highlights). The spherical harmonic light is only very low frequency. The spherical harmonic light does not produce a sharp illumination transition. They also affect scattered light only (the frequency of the spherical harmonic light is too low for the highlights).
• SH lighting is isn't local;  Point or spot SH lights close to some surface would "look wrong". Spherical harmony is not local, and the spherical and spot light and spotlight near the surface may "look incorrect".

In summary, SH lights is often good enough for small dynamic objects.

In general, the effect of spherical and light is good enough for small dynamic objects.

Details of deferred light rendering path Deferred Lighting Rendering path details

Deferred Lighting is rendering path with the most Lighting and shadow fidelity:

Delayed illumination is one of the most advanced rendering paths to enable light and shadow fidelity

• There ' s no limit how many lights can affect any object. There is no limit to the amount of light that can affect any object
• All lights is evaluated Per-pixel.  Which means that they all interact properly with normal maps etc. The full use of light is evaluated per pixel, which means that all will interact correctly with the object in the normal way of mapping
• All lights can has Cookies. All light can have an information cache
• All lights can has Shadows. All the light can create shadows.

Deferred Lighting ' s advantages benefits of delayed illumination:

• Lighting cost was proportional to light size on screen. Does not matter how many objects it shines on.  Small lights = cheap! The overhead of illumination is proportional to the light size of the screen, without worrying about the number of items that light shines, and a small amount of light equivalent to cheap spending
• Consistency. All lighting-lights is computed per-pixel;  There is no lighting computations that break off on large triangles etc. Uniformity, the illumination of all light is calculated by pixel as the calculated split unit. For example, there will not be a situation where the calculation performance crashes when the light is computed in a large-scale triangular case.

Disadvantages Disadvantages:

• No Real anti-aliasing support. No real-time anti-aliasing support
• Deferred Lighting can ' t handle semi-transparent objects.  Those is rendered using Forward Rendering. Delayed illumination cannot handle translucent objects, nor can they be used on objects that are rendered before they are used
• Limited Lighting Model Support (Blinn-phong). All lighting are computed the same;  You can ' t has drastically different lighting models on different objects. Limited illumination Mode support (Blinn-phong). All light is calculated in the same way that you cannot use a completely different light pattern on different objects.
• No support for "receive shadows" flags and limited support light culling masks. No support for receiving shadow features and limited support for light matte culling

Requirements for Deferred Lighting delay lighting Requirements

• Requires Unity Pro. Requires Unity Professional Edition
• Graphics card with Shader Model 3.0 (or later), support for Depth render textures and two-sided stencil buffer.     Most graphics cards made through 2004 support It:geforce FX and later, Radeon X1300 and later, Intel 965/gma X3100  and later. The display card supports shader Model 3.0 (or higher), deep texture rendering, and double-sided template buffering features. Many 2004 years later the graphics card is supported: such as GeForce FX or higher, Radeon X1300 or higher Intel 965/gma X3100 or higher
• Currently does not work on mobile platforms. Currently not supported on mobile platforms.

Performance Considerations Performance Considerations

Cost of realtime lights in Deferred Lighting are proportional to number of pixels the light shines on; and not dependent on scene complexity. So small point or spot lights is very cheap to render. Point or spot lights that is fully or partially occluded by some scene objects get their pixels skipped on the GPU, so th EY is even cheaper.

The overhead of real-time light in a delayed light is proportional to the number of pixel values illuminated by the light. Rather than depending on the complexity of the scene. Tiny point and spot light sources are very easy to render. Pixels illuminated by a point light source or a spot light source that is completely or partially obscured by a scene object are skipped by the GPU and are therefore much cheaper.

Of course, lights with shadows is much more expensive than lights without shadows. In Deferred Lighting, shadow casters still need to being rendered once or more for each shadow-casting light. and the lighting shader that applies shadows are also more expensive than one without shadows.

Of course, a light source with shadows is much more expensive than a light source without shadows. With delayed illumination, the light projector still needs to render one or more times for each shadow cast. Also, a shaded ray shader is much more expensive than a light shader that does not produce a shadow.

Implementation Details Implementation Detail

When Deferred Lighting was used, rendering process in Unity happens like this:

When delayed lighting is in effect, the rendering process that occurs in unity is as follows:

1. Base pass:objects is rendered, producing screen-space buffers with depth, normals, and specular power. Basic rendering: Rendered objects produce a screen space buffer with depth, normals, and reflections
2. Lighting pass:lighting is computed using the previous buffers.  Lighting is computed into another screen-space buffer. Illumination rendering: Uses the previous buffer to calculate the illumination. Result into another screen space cache
3. Final pass:objects is rendered again.  They fetch computed lighting, combine it with color textures and add any ambient/emissive lighting. Final render: Object is rendered again. Take the calculated light and color textures together, then add ambient light and scatter light.

Objects with shaders that can ' t handle Deferred Lighting was rendered after this process was done, using Rendertech-forward Rendering path.

Shaded objects that cannot use delayed lighting techniques are treated with a forward rendering path after the deferred light rendering is complete.

Base Pass Basic Rendering phase

Base pass renders each object once. View Space normals and specular power is rendered into single ARGB32 Render Texture (normals in RGB channels, specular PO Wer in A). If platform& Hardware supports reading Z buffer as a texture, then depth was not explicitly rendered. If Z buffer can ' t be accessed as a texture, then depth was rendered in additional rendering pass, using shader replacement.

Basic rendering renders each object once. The view space normals and highlight intensities are rendered into a single ARGB32 rendering texture (normals in the RGB channel, highlight intensity in a channel). If the platform and hardware support to read the z-buffer by texture, then the depth will not be explicitly rendered. If the z-buffer cannot be accessed in a textured manner, the depth will be rendered using shader substitution techniques in additional rendering processing.

Result of the base pass is Z buffer filled with scene contents and Render Texture with normals & specular power.

The result of the basic rendering is a Z-buffer filled with the contents of the screen and a rendered texture with normal and specular intensity.

Lighting Pass Illumination rendering phase

Lighting Pass computes Lighting based on depth, normals and specular power. Lighting is computed on screen space, so it's independent of scene complexity. Lighting buffer is a single ARGB32 Render Texture, with diffuse Lighting in RGB channels and monochrome specular Lighting in A Channel. Lighting values is encoded using logarithmic encoding to provide extended dynamic range than usually possible with ARGB32 Texture.

Light rendering calculates light based on depth, normals, and highlight intensity. Lighting is computed by the screen space and is therefore independent of screen complexity. The light buffer is a single ARGGB32 rendering texture, with the RGB channel of the texture having diffuse illumination information, with a single color-specific illumination in a channel. The illumination values are encoded in numerical order to produce a dynamically expanding range that can be achieved than the usual ARGB32 textures.

Lighting model is a fixed to blinn-phong.

The lighting mode is fixed to Blinn-phong.

Point and Spot lights This does not cross camera's near plane is rendered as 3D shapes, with Z buffer test against scene en abled. This makes partially or fully occluded point and Spot lights very cheap to render. Directional lights and point/spot lights that cross the near plane is rendered as fullscreen quads.

Point and Spotlight lights that cannot span adjacent planes are rendered as Z-buffered 3D shapes with an open test scene, which can be rendered very cheaply by partially and completely shielded point and spot lights. Parallel lights or point lights that span adjacent areas can be used as full-screen quads.

If a light has shadows enabled, they is rendered and applies in this pass as well. Note that Shadows is not "free"; Shadow casters need to is rendered and a more complex light shader needs to be applied.

If a light source with a shadow is in effect, it will be well rendered during this process. Note that shadows are not free, and shadow projectors require overhead to render, while a more complex ray shader needs to be applied.

Final Pass last render stage

Final pass produces final rendered image. Here all objects is rendered again; With shaders-fetch the lighting, combine it with textures and add any emissive lighting.

Lightmaps is also applied in the final pass. Close to the camera, realtime lighting are used, and only baked indirect lighting are added. This crossfades to fully baked lighting further away from the camera.

The final rendering phase produces the last rendered image, and all objects are rendered again, where the shader mixes the light and texture generated in the previous step, as well as all spontaneous light illumination.

The illumination map is also applied during the final render stage. Close to the camera, use real-time lighting, and bake only indirect lighting.

Other: Lightingmap baking stickers are not in the technical range of time. Baked light is a map that records lighting information to simulate a fixed lighting effect, and the scene itself does not contain a timely light.

Rendering Paths Comparison render path comparison

	Deferred Lighting delay Illumination	Forward Rendering Forward Rendering	Vertex lit vertex illumination
Features function
Per-pixel Lighting (normal maps, light cookies) calculates light per pixel (normal map, lighting cookies)	Yes	Yes	-
Realtime Shadows Real-time shadows	Yes	1 DirectionalLight (one parallel)	-
Dual lightmaps dual Illumination Map	Yes	-	-
Depth&normals buffers depth and normal buffer	Yes	Additional Render passes extra render channel	-
Soft particles Soft particles	Yes	-	-
Semitransparent Objects Translucent Objects	-	Yes	Yes
Anti-aliasing anti-aliasing	-	Yes	Yes
Light culling masks lighting rejection Mask	Limited	Yes	Yes
Lighting Fidelity Illumination Fidelity	All per-pixel pixels	Some per-pixel some pixels	All Per-vertex vertices
Performance performance
Cost of a per-pixel light per pixel	Number of pixels it illuminates the number of pixels illuminated	Number of pixels * Number of objects it illuminates number of pixels * illuminated pixels	-
Platform Support Platform
PC (WINDOWS/MAC) Desktop	Shader Model 3.0+	Shader Model 2.0+	Anything
mobile device (ios/android)	-	OpenGL ES 2.0	OpenGL ES 2.0 & 1.1
Consoles (game) platform	PS3,	PS3,	-

Introduction of three-time illumination rendering methods in game engine (take Unity3d as an example)