Image Based lighting and physical shading in Iron Man and other movies

Http://renderwonk.com/publications/s2010-shading-course/snow/sigg2010_physhadcourse_ILM_slides.compressed.pdf

Http://renderwonk.com/publications/s2010-shading-course/snow/sigg2010_physhadcourse_ILM.pdf

Siggraph10 mentioned in physically based lighting course.

Brought by Industrial Light and magic, the original title is image based lighting and physical shading in Iron Man and terminator, but in fact this paper covers IML's years of accumulation, including Pearl Harbor and Hulk.

At the same time, there are many practices in it, which are also very instructive for realtime graphics.

Film reality

One thing to be clarified is that the ultimate goal of CG developers is film reality, which is not the reality visible to the human eye. This restriction comes from the fact that the storage medium is LDR, so under this restriction, many things need to consider these factors.

1990s

In 1990s, we captured an LDR environment texture (LDR at that time, and HDR later) for image based lighting. At that time, we used Cook Torrence for specular formula.

Some hack: Typical is some ambient lighting, which cannot be very accurate for capture and generate shadow information. A large amount of Spotlight will be used for simulation:

(Irradiance volume)

Pearl Harbor

Michael Bay (the director of "brave island" Pearl Harbor "Transformers") challenged IML to say that it would be more realistic than CG rendering in previous movies, as a result, we all saw that we thought it was a real-time airplane. Just like the war, we were able to push forward technological progress more than 10 universities. The demand was really the best fuel aid for technological progress.

By the end of 1990s, raytracing (ray tracing) technology was quite mature, but IML was still not used. It was due to the fact that it was not practical to use in such complex scenarios and the software they were used to (renderman) does not yet have this function (well, good hands-on experience), so continue with their image based lighting.

Added

• Reflection occlusion: adds an occlusion to the illumination Information read from the image to prevent a dark and bright reflection environment from being located in the blurred area.
• 
  
• Ambient Environment
• 
  

To achieve the GI quality close to raytracing with low consumption of resources at hand (Development consumption and calculation consumption of generating actual images. Finally, we achieved this quality: (coooool)

Hulk (<Hulk>)

A progress has been made to the Hulk, that is, the environment map is changed to HDR, and the resolution is further increased.

Iron Man

Here Iron Man is mainly dealing with materials. The work on environment pasters and other image based lighting is quite mature (stable use of high-resolution HDR environment pasters ), however, iron's high material requirements posed new challenges, and soon the project team found that BRDF on materials was not good enough.

Here, we must mention the following: we can see that there is a great final result, how much accumulated work needs to be done in the middle, and a lot of their work will not even be exposed in the final product.

• In transformers, ILM has successfully made a lot of materials suitable for cars, paving the way for Iron Man's work.
• ILM uses the BRDF detector (recording how the material reflects light, Bungie has done this in Halo ),
• However, it was found that the chrome alloy was too complex to collect enough information.
• The collected information is great, but it is very difficult for art to make some changes (it is estimated that the inexplicable parameters of a large ticket are directly dizzy)
• Finally, the test BRDF was not used, but it helped the project team better understand the BRDF of the material.

Based on these analyses, ILM uses the specular function of the opposite sex for the cool metal effect:

There are also a lot of research on the BRDF of the opposite sex. ILM uses different specular exponent in some vector directions.

Use brickmap for Ray traced reflection surface caching.

ILM encountered many problems during the development of iron Xia, one of which is many parameters (influencing factors), which are too complicated. After a fine arts call, there are many differences between people, at the same time, there will also be unrealistic situations. Simplification and other problems have made a lot of progress in the development of Terminator.

Here, ILM is quite good: ambient and diffuse are actually one thing, while reflectance and specular are actually one thing.

Terminator

In Terminator: salvation, ILM hopes to solve some headaches in previous project development (such as the dilemma of too many parameter adjustments ), and meet new challenges (the changes in some Terminator scenarios are more intense ).

ILM uses physically based lighting systems:

• Achieve a simpler, more intuitive, and more physical lighting Rendering System
• This includes:
• Energy conservation material
• Normalized specular
• More physically plausible specular falloff
• More heavily image based
• Importance sampled raytracing

Conservation of energy is a basic feature of physical light. It ensures that the reflected light intensity does not exceed the incident light. At the same time, ILM merges ambient and diffuse, specular and reflectance, so that we only need to balance diffuse and specular (unlike the four items originally to be balanced ).
Normalized specular: if a light is reflected on a rough surface, the reflected surface will be larger and the brightness will be reduced. In fact, this is also a kind of conservation of energy. It turns out that ILM does not have this item. It is manually adjusted by people. Now, in the specular formula, the results are more natural without being adjusted.

Normalized importance sample raytracing

The importance sample is the main part of the De-sampling to be as effective as possible.

This reminds me of the reflection in the sermaria ue3 people, which only reflects the neon lights, and the buildings do not care about this. This is also a kind of importance sample.

In actual application, the importance sample includes:

• Mainly samples brighter points in the image.
• When the surface is rough, the direction of reflection is changed to the most effective part.

The final result is similar to the standard raytracing, but the efficiency is much better.

Troubles and gains of change
Following these technological changes, ILM, like many companies, has the following pain points:

• For the impact on production, ILM is also developing and making movies, so it follows the original system to make a good part of the original system that can be processed, such as the motorcycle robot inside, then, after the system is developed, it will be challenging, such as the skeleton robot.
• The initial technological changes have also led to tool efficiency issues (if bugs and crash need to be added to the game), the efficiency of the tools to be prepared is quite low (compared with the original)
• Art artists who used to use the original hack are also quite unaccustomed to the use of the new system. ILM has even triggered a "war", in short, it takes a lot of effort to implement the new system (such as energy conservation and normalized specular)
• However, at the end of the project ("Iron Xia 2"), the pain period passed, and the entire line was transferred to a new physical material lighting system, with higher quality and development efficiency.

In the end, iron Xia shot the image like this. I feel the actor won't be very high when shooting it: