[Shader expansion] implements strative rendering in team Fortress 2

Preface

As early as the use of ramp texture to control diffuse illumination, this famous paper was mentioned. Other results published by valve can be seen here. This is a very influential article published by valve in 2007. My mentor also mentioned this article. Since it is so famous, I went to read it. The result is really big! In fact, I read this article half a year ago and almost finished reading it. Later, I was not finished after I had a day off, so I am very sorry for myself... After a long time, I hope I can sort out my previous understanding and see if I can get more results. (Read the e file again !)

Based on the translation of this paper, this blog will try its own effect in Unity shader. If there are any discrepancies, please also point out, thank you very much.

Summary

In 2007, a video-quality game like Fortress 2 was rare. This paper illustrates the shader technology used in the game. A good game must be a perfect collaboration between artists and engineers, so this paper will discuss how they use technology to support the art goals and game restrictions. Furthermore, to get engaging results, their shading Technology uses rim highlights and changes in brightness and color to quickly deliver different geometric information.

Keywords: Non-Realistic Rendering (rendering effects a bit like cartoons or illustrations), lighting models, pasters, hardware rendering, and video games

1. Introduction

Before that, most non-Realistic Rendering technologies can only work under a specific model. This article introduces a series of interactive rendering technologies, it also adds changes in light and color, so that players can visually distinguish different scenes under different lighting conditions.

In Fortress 2, they chose the artistic style of several illustrators in the middle of the 20th century. This style of illustration role uses a strong and clear outline and emphasizes clothing folds. They increase the internal shape of the object, using edge highlights rather than black outlines to emphasize the object contour, as in (.

The main contribution of this paper is the transformation from the Commercial Illustration style to the game. Technologies include the implementation of a diffuse light warping function illustrator rendering function, the rim lighting formula, and a shading Technology that balances the real and non-realistic aspects in general.

2. Related Work

Non-Realistic Rendering styles vary, but they are all artistic styles from the real world, provided that these technologies are developed by humans and are inherently valuable. The Commercial Illustration style used in Fortress 2 is closely related to Gooch in 1998. In Gooch shading, the traditional Phong Model [Phong 1975] is modified by a cold to warm color change to imply a plane direction under a certain light source. Therefore, very bright and dark areas are retained as edge lines and highlights. Compared with the traditional lighting model, this method achieves clearer 3D vision under some difficult lighting conditions.

3. Commercial Illustration skills

According to the illustrator J. C. leyendecker, Dean Cornwell, Norman Rockwell and Their Own Original Concept in the early 20th century, they achieved the following consistency to define the style in Fortress Legion 2:

• Shading follows a warm to cold color conversion. Shadows tend to be cold rather than black.
• In the light and shade alternation of a specific light source, the saturation increases. The alternate place is usually red.
• The details of the high frequency are omitted as much as possible.
• For a role, internal details, such as clothing folds, and so on, repeat contour lines.
• Highlight the contour with an edge rather than a dark line.

Based on the above basic understanding, they have created art resources and real-time coloring algorithms.
The next section describes how to create art resources. The technical details are described in section 5.

4. Create art resources

This section describes the modeling of 3D figures and world models, as well as the specifications that are followed when textures are generated.

4.1 character Modeling

In multiplayer Combat games like Legion Fortress 2, we need to be visually able to quickly identify different characters at various distances and perspectives to assess threats. In Legion Fortress 2, the player categories-demo, engineer, heavy, medic, Pyro, spy, sniper, soldier or scout-are particularly important, these nine types of outlines must be carefully designed to look very different from each other. As shown in.

The body proportions, weapons, and contour lines determined by the folds of shoes, hats, and clothes give each role a unique profile. In the covered interior area of the role, the clothing folds and repeats the contour lines to emphasize the outline, as we observed in the commercial illustration. In section 5th, the coloring algorithms used for these roles complement our model to increase the shadow ratio.

4.2 Environment Modeling

The environment in legion Fortress 2 is related to two teams: blue and red. Therefore, the two teams are defined with distinctive characteristics. Red teams use warm colors, natural materials, and edges. The blue team mainly uses Cold colors, industrial materials, and orthogonal graphics. As shown in.

Although the actual modeling has more details, they try to avoid complicated or geometric imbalance models, because it will increase unnecessary visual noise and the vertex that will burden the memory. Furthermore, the use of repetitive structures, such as cable trays, Poles, and railway tracks, to a minimum is more suitable for our style, because the feeling of repetitive space is more important than expressing every detail.

4.3 texture painting

In fortress Legion 2, the color of the role and game is stronger than that of the real material, and the color saturation and contrast are stronger. The red and blue teams each have their own tone in the game, as shown in their reference sample color. We can see that most of them use soft colors.

In addition to the two main red and blue colors, other colors are used for environmental items in some small areas, such as fire extinguishers and telephones. Normally, textures used in 3D environments are post-visual, meaning they are painting-style and maintain minimal visual noise. This style is used in many animated movies, especially those of Hayao Miyazaki. In these works, there are a lot of strokes in the perspective. We use the same technology for our 3D games.

A large number of environment textures come from hand-drawn reflection textures, which have a small amount of internal details and use large strokes to express the touch of a specific plane, as shown in. In early development, many of these 2D textures were completed using watercolor and scanning on the canvas. Later, artists turned to realistic reference photos and used a series of filters and digital brushes to get desired textures.

5 interactive character and model Shading

This section describes non-realistic shadow algorithms. For character roles and most models, we integrate a series of dependency and non-dependency perspectives. As shown in.

The non-dependent view part is composed of a parallel environment light part with a space change plus a modified Lambert light part. The dependent view consists of the Phong highlight and custom edge illumination. All of these illumination parts are calculated on a pixel-by-pixel basis, and most material properties, including normal, reflectivity, mirror reflection, and various such masks, are produced by texture sampling. In the following two chapters, we will discuss how different lighting components and traditional methods affect our performance.

5.1 non-dependent illumination

The non-dependent view part is composed of a parallel environment light part with a space change plus a modified Lambert light part. It can be expressed by the following equation (1:

Here, l is the number of light sources, I is the light source index, is the color of light source I, is the reflectivity of the object mapped by textures (albedo ), it is a traditional unclamped lambertian term about the light source I. constants α, β, and gamma are the contraction, offset, and exponent of lambertian term, respectively, it is a function to evaluate the optical part of a parallel environment for the normal n of each pixel, and w () is a deformation function that maps scalar values in the 0-1 range to an RGB color.

Half Lambert)The first unusual part of equation 1 is the scaled down, offset, and exponential Part applied. Because our first game "Half Life", we used 0.5 times of scale-down, 0.5 offset, and square to prevent the role from losing its shape in the backlight (α = 0.5, β = 0.5, Gamma = 2 ). Although more realism is needed in the current game, we still use these settings to convert the result of vertex multiplication (range:-1 to + 1) to 0 to 1, and there is a satisfactory attenuation zone. Thanks to the 0.5-fold reduction of the lanbert part and the offset of 0.5, we call this technology "semi-lanbert ". In Legion Fortress 2, we make alpha = 0.5, Beta = 0.5, but Gamma = 1, because we can get any desired shadow through the W () function.

Diffuse warping Function)The second interesting thing in equation 1 is the deformation function W (). The goal of this function is to keep the shadow information of semi-lanbert while introducing the beautiful light and shade handoffs in the commercial illustration. In Legion Fortress 2, this deformation function uses a one-dimensional texture for search ing, as shown in. This method indirectly creates a "hard shadow". While retaining all normal changes to the light, we tighten the hop from the light to the dark. For example, 6 (B ).

Note:As can be seen from the figure, the variation in the alternating light and shade is fast, so the tightening is achieved.

In addition to the common "shadow", this one-dimensional deformation image has some interesting features. First, the rightmost value is not white, but a slightly brighter color than mid-gray. This is because after finding this texture, you still need to multiply it by 2 (this will make it brighter), so that art personnel can better control the brightness and shade. It is also important. This texture is divided into three parts: the gray gradient on the right, the cold gradient on the left, and the small red line in the middle. This is consistent with the changes in the shadow of the illustration we observed earlier, that is, it tends to be cold rather than black, and it is usually a little reddish in the decomposition. As shown in equation 1, after the deformation function acts on the half-lanbert factor, an RGB color is obtained and then multiplied by the fine-tuned color (the color of the light source) to obtain a diffuse reflection part, 6B.

Directional ambient term)In addition to the sum of simple deformation and diffuse reflection illumination, we also applied a parallel environment illumination section ,. Although the representation is different, our parallel environment light is equivalent to an environment radiation illumination ing (inrradiance environment map, [ramamoorthi and Hanrahan 2001. However, we did not use a nine-term spherical tuning formula (a 9 term Spherical Harmonic basis), but used a modified six formulas, we call it the ambient cube, which uses the cosine square extending along the X, Y, and Z positive and negative directions to implement [mctag-Gart 2004] [Mitchell et al. 2006]. We use offline radiosity solver to compute these environmental boxes and store them in an irradiance volume for real-time access to [Greger et al. 1998]. Although this illumination part is simple, as shown in 6C, This Ambient Light Box provides information for reflecting light, which is the key basis for rendering the characters and other models in the game.

The final result of adding the illumination parts unrelated to the angle of view is 6d, and then multiplied by the color reflectivity (albedo, figure 6a) of the model, as shown in 6e.

5.2 illumination part dependent on the perspective

The illumination part of our dependent view is composed of a traditional phony mirror Highlight model and a custom edge highlight part. Equation (2) is as follows:

Here, l is the number of light sources, I is the index of light source, is the color of light source I, is the mirror highlight mask (specular mask) embedded into a texture channel, is the angle vector, fresnel is a Fresnel factor adjusted by art for the mirror highlight. It is the reflection vector relative to the direction vector of the light source I, is the up vector in the world coordinate system. It is an exponential part of a mirror reflection mapped by a texture. It is a constant index, used to control the width of the edge highlight (the smaller the width), is another fresh factor, used to modify the edge highlight (usually used), is an edge mask texture (rim mask texture ), it is used to reduce the influence of the edge highlight on some parts of the model. Finally, it is an evaluation of the environment box (ambient cube) using the observation direction (I am not sure about the last sentence... Original article: is an evaluation of the ambient cube using a Ray from the eye through the pixel being rendered ).

Multiple Phong term)The left half of equation (2) contains a common expression -- used to calculate the Phong highlight, and adjusted it using an appropriate constant and a fresh factor. However, we also use the max () function within the sum to combine the Phong highlight with the additional Phong lobes (using different exponent, fresh coefficient, and mask. In Fortress 2, a model is a constant and much smaller. The light source generates an edge highlight profile, which is not affected by the material attributes. We used a fresh coefficient to mask these edge highlights to ensure that they only appear at the tangent angle (grazing angles. This combination of Phong highlight and edge highlight makes Legion Fortress 2 very good.


Dedicated edge lighting (dedicated rim lighting)When the role gradually moves away from the light source, only the edge highlight Based on the Phong part may not be as obvious as we want. To this end, we have added a dedicated edge highlight section (the right half of equation 2 ). This section uses the observation direction to evaluate the environment box (ambient cube), and uses an art-controlled mask texture, the fresh factor, and the expression for adjustment. The final expression is only the result of multiplication of the pixel-by-pixel normal and the up vector of space (up vector) point, and then constrained to the positive number range (clamped to be positive ). This makes this specialized edge highlight look like an indirect light source in the environment, but it is only applicable to face-up normal. This method is an aesthetic and perceptual choice, and we hope this will make people feel that the illumination is from the top.

The complete pixel shader used for role and other model rendering is the sum of equations (1) and equations (2), and some other operations similar to the ing of optional environments.

Practice

I tried the above process using Vertex & fragment shader in unity. Here, we only consider a single light source, and do not process the parallel ambient light in a non-dependent light source and the ambient light box in a dependent light source. The following is the processing result (for reference only ):

Finally, the sample code for using the above model in Unity shader is provided (just an example of the formula! Only a single light source is processed !) :

Shader "Custom/IllustrativeRenderingNormal" {Properties {_MainTex ("Base (RGB)", 2D) = "white" {}_NormalTex ("Normal Texture", 2D) = "white" {}_RampTex ("Ramp Texture", 2D) = "white" {}_SpecularMask ("Specular Mask", 2D) = "white" {}_Specular ("Speculr Exponent", Range(0.1, 128)) = 128_RimMask ("Rim Mask", 2D) = "white" {}_Rim ("Rim Exponent", Range(0.1, 8)) = 1}SubShader {Pass {Name "FORWARD"Tags { "LightMode" = "ForwardBase" }CGPROGRAM#pragma vertex vert#pragma fragment frag#include "UnityCG.cginc"#include "Lighting.cginc"#include "AutoLight.cginc"sampler2D _MainTex;sampler2D _NormalTex;sampler2D _RampTex;sampler2D _SpecularMask;float _Specular;sampler2D _RimMask;float _Rim;float4 _MainTex_ST;float4 _NormalTex_ST;float4 _SpecularMask_ST;float4 _RimMask_ST;struct v2f {float4 position : SV_POSITION;  float2 uv0 : TEXCOORD0;  float2 uv1 : TEXCOORD1;  float2 uv2 : TEXCOORD2;  float2 uv3 : TEXCOORD3;  float3 viewDir : TEXCOORD4;  float3 lightDir : TEXCOORD5;};v2f vert(appdata_full v) {v2f o;  o.position = mul (UNITY_MATRIX_MVP, v.vertex);o.uv0 = TRANSFORM_TEX (v.texcoord, _MainTex); o.uv1 = TRANSFORM_TEX (v.texcoord, _NormalTex); o.uv2 = TRANSFORM_TEX (v.texcoord, _SpecularMask); o.uv3 = TRANSFORM_TEX (v.texcoord, _RimMask); TANGENT_SPACE_ROTATION;  float3 lightDir = mul (rotation, ObjSpaceLightDir(v.vertex));  o.lightDir = normalize(lightDir);float3 viewDirForLight = mul (rotation, ObjSpaceViewDir(v.vertex));  o.viewDir = normalize(viewDirForLight);// pass lighting information to pixel shaderTRANSFER_VERTEX_TO_FRAGMENT(o);return o;}fixed4 frag (v2f i) : COLOR {half3 normal = UnpackNormal(tex2D (_NormalTex, i.uv1)); // Compute View Independent Lighting Termshalf3 k = tex2D( _MainTex, i.uv0).rgb;half difLight = dot (normal, i.lightDir);half halfLambert = pow(0.5 * difLight + 0.5, 1);half3 ramp = tex2D(_RampTex, float2(halfLambert)).rgb;half3 difWarping = ramp * 2; // Or difWarping = ramp * 2;half3 difLightTerm = _LightColor0.rgb * difWarping; half3 dirLightTerm = 0;half3 viewIndependentLightTerms = k * (dirLightTerm + difLightTerm);// Compute View Dependent Lighting Termshalf3 r = reflect(i.lightDir, normal);half3 refl = dot(i.viewDir, r);half fresnelForSpecular = 1; // Just for examplehalf fresnelForRim = pow(1 - dot(normal, i.viewDir), 4);half3 kS = tex2D( _SpecularMask, i.uv2).rgb;half3 multiplePhongTerms =  _LightColor0.rgb * kS * max(fresnelForSpecular * pow(refl, _Specular), fresnelForRim * pow(refl, _Rim));half3 kR = tex2D( _RimMask, i.uv3).rgb;half3 aV = float(1);half3 dedicatedRimLighting = dot(normal, half3(0, 1, 0)) * fresnelForRim * kR * aV;half3 viewDependentLightTerms = multiplePhongTerms + dedicatedRimLighting;                  // Compute the final color         float4 col;         col.rgb = viewIndependentLightTerms + viewDependentLightTerms;         col.a = 1.0;                  return col;}ENDCG}}FallBack "Diffuse"}

[Shader expansion] implements strative rendering in team Fortress 2