11th dynamic combination of loose shader

Shader is very strange.CodeIts length is limited, its dynamic branch capabilities are weak, and its commands are expensive, which makes it difficult for you to use a single shader to handle all rendering requirements. there are so many different types of rendering requirements. If you want to write a specific piece of code for each rendering type, it will be very difficult, if we want to write a material system, we hope it can support various effects. you will find that as the support effect increases, the number of shader to be written will rise sharply, for example: * At first we wanted our material system to support normal lighting. We wrote a shader for the simplest effect. *. Then we hope that this material can also support textures. Let's write another shader for it. Now there are two shader -- [Supports normal illumination, not textures] and [Supports normal illumination, supports textures] *. Then we want to add a skeleton animation for it. We need to write a version with a skeleton animation for each existing shader, so that the number of shader will become four, respectively: [Supports normal lighting, does not support textures, and does not support bone animation] [Supports normal lighting, textures, and bone animation] [Supports normal lighting, textures, and does not support bone animation] [Supports normal lighting, does not support textures, and supports bone animation] *. Then we want to add fog. We need to write a version that supports fog for each existing shader. In this way, the number of shader is changed to 8, respectively: [Supports normal lighting, does not support textures, does not support bone animation, does not support Fog] [Supports normal lighting, textures, bone animation, and Fog] [Supports normal lighting, textures, does not support bone animation, and does not support Fog] [Supports normal lighting, does not support textures, supports bone animation, and does not support Fog] [Supports normal lighting, textures, skeleton animation, and fog.] [Supports normal lighting, textures, bone animation, and Fog] [Supports normal lighting, textures, does not support bone animation, and Fog] [Supports normal lighting, does not support textures, supports bone animation, and supports fog] You can note that when we add a new function, the number of shader will double. when we keep adding new features, this number will soon become intolerable. writing each shader manually will cause a crash. One solution is to write a shader that supports everything. [supports common lighting, textures, skeleton animation, and Fog], then, we can set different parameters to achieve the correct effect. For example, if we do not need a texture, we can set a white texture. If we do not need a skeleton animation, we can set an array of identity matrices. If Fog is not required, we can set the start distance of fog to a large value. in this way, we only need to write a single shader with powerful functions, but as mentioned above, the shader code has various restrictions, so that a single shader will become very slow as the function increases more and more, if this slow shader does complete many functions, it is acceptable, but when it completes only a small subset of all functions, it still takes the same time, this is unacceptable. it is said that the new generation of video card and runtime have greatly improved in this regard, but I have never used it and I don't know what will happen. I think it is still awkward. in any case, our engine is for DX9 and we must find another way to solve this problem. Using predefined macros can better solve this problem. We can write a long shader file containing various functions, then, a large number of # ifdef/# else/# endif functions are used to separate them. Then, when using shader, we compile the shader file by specifying a combination of different predefined macros to get a shader with the correct function cropping. since we cannot complete dynamic branches within the shader, we can only let the compiler help us compile the branches into static shader. In our engine, a very similar method is used. The following describes the specific implementation: First, let's talk about my understanding of shader computing. although the shader code has various bad qualities, its final calculation result is simple. In most cases, it is only a float4 color, the final result can be calculated by some intermediate results, which can be calculated by a second intermediate result. I call the final calculation result output, the intermediate result is called factor, and the calculation process is called formular. A typical shader can be described in the following form. The following describes the specific implementation: *. First, we have the concept of a shader Library Project, which is somewhat similar to a project in VC. It consists of several files, and its build result is a shader library.

*. A shader library is not a shader. It contains many shader functions. We can generate corresponding shader from this library based on the combination of various functions. *. A shader library project must contain only one template file. The template file defines all the factors used in this library (note all the factors ), and some default formula, such: Note that this token does not contain shaderconstant, and each formula can freely access all shaderconstant, so I will not draw it out. *. In addition to templates, the project also contains several feature files. A feature represents a function to be implemented by shader, such as supporting textures and bone animation. One feature defines one or more formula, they are all the heavy-duty versions of formula defined in the template file. for example:

Feature dirlight reloads a formula:

Function diffusemap reloads a formula:

Feature bones reloads two formula:

Function fog reloads two Formula

*. With the template file and function file, we can combine the shader. First, we specify the function combination we need, then we simply need to replace formula, which is reloaded with the function file, with the corresponding version in the template file. We will get a new computing process, such:

Then we generate the corresponding shader code based on this process and hand it to d3dx for compilation (currently we use d3dx's effect system to use shader). Then we can get a combined shader. Notes: *. The preceding graphical representation is actually only one, which is easy for readers to understand. Currently, all template files and feature files in the engine are written in scripts, their processing involves a lot of syntax analysis and String concatenation work, and there is no dedicated Graphical Editor. I hope to have the opportunity to update it to a real graphical editing method in the future. *. For some performance considerations, all feature files are defined in the C ++ header file. Each feature is represented by one bit. We use an unsigned int64 file to represent the combination of various feature files. that is to say, we can define a maximum of 64 different feature types. (We thought it was enough when we designed this system. However, we need some trick to deal with the useful trend .) *. There are some feature that cannot coexist. For example, there are four feature about skin: bones1, bones2, bones3, and bones4, which correspond to the condition that the same vertex is affected by the number of different bones, they cannot coexist. We have added support for the conflict group in the template file to store mutually exclusive feature in the same conflict group, we will detect feature conflicts based on conflictgroup when combining shader. *. Multiple feature that can coexist may overload the same formula. We need to specify a priority when reloading formula to solve this problem, multiple feature uses this priority to compete for the same formula. *. Using this system only facilitates the shader writing process to a certain extent. Designing a template file is still very complicated and requires sufficient experience in shader programming, and overall ability to grasp. Using this system to dynamically assemble shader is essentially the same as using # ifdef/# else/# endif, but it has some advantages: *. The template file records only some key nodes in the entire computing process, and it will not become very long, using # ifdef/# else/# endif can easily lead to writing a large file, and the main computing flow will be drowned in various details. *. The code of the same feature can be written in the same file in a centralized manner, without being dispersed in various places of a large file, which is easy to maintain. * The feature file can be shared by multiple shader projects, as long as the formula. *. Because the shader combination is completed by ourselves, we can present the combination result (a standard HLSL file) to users for inspection. I don't know if the HLSL compiler provided by d3dx has similar functions. The dynamic combination of shader includes the combination of HLSL Code (which is completed by US) and compilation of HLSL Code (which is completed by d3dx), which is a relatively slow process, we obviously cannot repeat this process every time we use a shader. We will save every combined shader in a cache, then, use the shader library name + 64-bit feature combination code as the hash key to index it. this cache will be saved in a file and loaded every time the game is running. Let's talk about the dynamic combination of shader. the specific use of shader is still messy. Let's talk about it next time.