How to separate highlights and diffuse information from real photos

Original address: http://filmicgames.com/archives/233

In the previous article, where all objects are shiny, the author of this article will show how to separate diffuse and specular information from the picture.

You may think that the next few things will be difficult, but it doesn't really matter what you think of you. One thing I do at home is to use polarized light to separate diffuse and specular highlights from real images. Artwork (Artists) spends a lot of time looking at the reference picture, but it helps you understand what a material is by separating diffuse and high energy. In the illustration above, the left side is only diffuse light, with diffuse and specular highlights in the middle, and only highlights on the right. By the way, I'm not an expert on this, but that's how I understand it.

1. original image:

2. Diffuse:

3. highlight:

   If you want to do the separation yourself, you need to:

4. A camera that can be photographed manually. Like a DSLR.
5. a remote shutter controller so you can shoot multiple photos without touching the camera.
6. A light source, I used an Ikea lamp.
7. some polarizers. I purchased it from Polarization.com. You need to "fully cascade linear polarizers". Make sure you're using a linear polarizer, not a round polarizer.

   At the most basic level, in graphics we assume that objects have diffuse specular highlights. is a light source that shines on the surface and has a camera next to it. By the way, this model is a very large simplification.

   Where some light reaching the surface leaves the surface. If there is a white light source and a blue surface, then the reflected light is still white, and we call it the highlight.

   Some other light will be absorbed by the surface, some electrons will be activated, and a new photon will be emitted in a random direction. If we have a white light source and a blue surface, then the shot light will be blue and we call it diffuse.

   I hope you understand that. If you don't know it before, you should know it now. One thing that's particularly cool is that light can be polarized. You can from your good friend Wikipedia (domestic please Baidu or Google polarization) in addition, the high light will keep it in the polarization, but diffuse reflection will not. Because diffuse light is re-propagated in a random direction and is accompanied by random polarization, we can say that it is non-polarized.

   The following are the required settings:

   It's easy, I've got all the cameras in the manual mode, which means:

1. Manual exposure
2. Manual aperture Adjustment
3. Manually adjust the white balance
4. Manually adjust the ISO
5. Turn off autofocus
6. Ensure that the color configuration is sRGB and not Adobe 98

   You also need to have a remote shutter controller. This allows you to take multiple photos without touching the camera. If you have to touch the camera to shoot, then I am sure there is a situation of misalignment. And we need a light source, let's take a look at it.

   It is an IKEA halogen lamp, but a polarizer is placed in front of it. I use tape to fix it. As a warning, the lantern is very hot, so the polarization is a little distorted.

   All we have to do is take two pictures. But I will need to put a polarizer in front of the camera. Note that there is a polarizer on our camera and light source. If you think I'm a game programmer from Los Angeles, then you're wrong. It's just that I used to pay the rent, when I wanted to be a hand model dream (joke).

   Notice the orange tape on the polarizer? I'm going to rotate it 90 degrees and take another one. Look, the orange tape has moved.

   Here's the look of our first photo shoot. If your alignment is perfect, then there should be no highlight information. Of course, there will be a little bit, because it is basically impossible to align the polarizer by hand.

   How does it work? The polarized light reaches the surface and the highlight bounces off the surface and shoots into the camera. It reaches the polarizer perpendicular to the polarized light, so it is completely absorbed. At the same time, diffuse absorption and re-transmission of reflected light is non-polarized. Polarized light absorbs half of the diffuse sunlight, and the rest of it enters the camera. So there's half the diffuse but no highlight information in the picture. Now look at the second picture.

   In the above picture, the polarized high light reaches the polarizer parallel to it, so all the light is passed. At the same time diffuse reflection is not non-polarized, all half of it is absorbed. It has half the diffuse reflection and all the highlights information.

   If we contract the first picture to a, and the second one to B, then diffuse is 2*a, and the high light is b-a. Of course these images are stored in the sRGB space. So here we need to use the shader code to compare the two images and separate them and store the results as an sRGB image. As always, the code has not been tested.

　　　　　　

1 floatLineartosrgb (floatval)2 {3    floatret;4    if(Val <=0.0)5RET =0.0f;6    Else if(Val <=0.0031308f)7RET =12.92f*Val;8    Else if(Val <=1.0f)9ret = (Pow (val,0.41666)*1.055f)-0.055f;Ten    Else OneRET =1.0f; A    returnret; - } -  the floatSrgbtolinear (floatval) - { -    floatret; -    if(Val <=0.0f) +RET =0; -    Else if(Val <=0.04045f) +ret = val/12.92f; A    Else if(Val <=1.0f) atret = POW (val +0.055f)/1.055f,2.4f); -    Else -RET =1.0f; -    returnret; - } -  in intG_bspecordiff; -  to float4 Ps_main (float2 texcoord:texcoord0): COLOR + { -FLOAT3 Srca =tex2d (TEXTURE0, Texcoord). RGB; theFLOAT3 SrcB =tex2d (Texture1, Texcoord). RGB; *FLOAT3 LinA =srgbtolinear (SRCA); $FLOAT3 LINB =srgbtolinear (SrcB);Panax NotoginsengFLOAT3 Lindiff = lina*2; -FLOAT3 Linspec = linb-LinA; theFLOAT3 Texdiff =Lineartosrgb (Lindiff); +FLOAT3 Texspec =Lineartosrgb (linspec); AFLOAT3 ret = G_bspecordiff?Texdiff:texspec; the    returnret; +}

If you do all the right things above, then the diffuse picture should look like this.

The highlight picture should look like this:

As you can see, the process is not perfect.

1. There is some highlight information in the diffuse picture.
2. Some objects do not behave very well. In the only high-gloss picture, the handle is slightly bluish. I think the handle is a non-conductor causing some disturbance of frequency.
3. If highlights or diffuse reflections are truncated to 1.0, then these pixels are wrong. It is better to be exposed than overexposed.
4. Make sure you use a remote shutter controller, or you will have alignment issues.

Hopefully not too much trouble, you can separate the highlights and diffuse reflections yourself.

How to separate highlights and diffuse information from real photos