Depth Camera (ii)--structure light depth ranging __ stereo vision

Original:

Http://blog.sina.com.cn/s/blog_80ce3a550100wg5j.html

http://blog.csdn.net/u013360881/article/details/51395427

Online resources:
Http://eia.udg.es/~qsalvi/recerca.html

Structured Light Code:

In 3D depth acquisition, the most common method is similar to the binocular matching to get the depth of the method, the binocular matching is based on the image processing technology, and the matching point is obtained by looking for the same feature points in two images, which can get the depth value, and the method based on image matching is very difficult, and the accuracy and correctness of matching are difficult to guarantee. Therefore, structured light technology is used to solve the matching problem.

Compared with the normal binocular ranging:
In the common binocular ranging, the light source is the ambient light or white light, which has no coded light source, and the image recognition depends entirely on the feature points of the object being photographed, so the matching is always a difficult point of the binocular, and the difference of the structured light ranging lies in encoding or characterization of the projection light source. In this way, a coded light source is projected onto an object that has been modulated by the depth of the object's surface, because the structure light source has many feature points or codes, so it provides a lot of matching corner points or direct code words, which can easily match the feature points, In other words, structured light offers many feature points to match or directly provide code words, instead of using the feature points of the subject itself, so it can provide better matching results.

The difference in the match:

The binocular uses the characteristic point of the object itself, And the structure light uses the characteristic point which the light source offers voluntarily or the direct code word. Another difference is that because of the variety of shooting objects, each binocular matching face different images, need to extract feature points, and the structure of light projection is the same pattern, feature points are fixed, do not need to change according to the scene changes, reduce the difficulty of matching.
According to the law of sampling, it can be considered that the camera takes the projection structure light a sampling process, so it must satisfy the sampling requirement of twice times frequency, which causes the resolution of the image to be taken higher than the resolution of the projected structure light image (horizontal, vertical resolution is twice times);
The theoretical basis of structured light ranging: (from calibration to explain, the basic principle of binocular ranging), reference paper "Review of Structured Light coding method"

It is assumed that the camera coordinate system is the world coordinate system, and the camera and the object are completely parallel, that is, the x,y coordinates of the object in the world coordinate system and the x,y in the camera coordinate system are the same, so there is no outer parameter matrix of r,t. The coordinates of the plane coordinate system can be obtained by simplifying the mapping of the above parameters and the world coordinate system without considering Z. However, the structural light projection equipment does not meet the assumption of the camera, so there exists r| between the projection equipment and the space point (x,y,z). The outer parameter of T. Given the projection device and the internal parameter matrix of the camera, moreover, the external parameters are calculated in advance by a general calibration algorithm, then the value of z can be computed by the mapping Point (XG,YG) of the space point on the projection device and the mapping Point (Xi,yi) on the camera image plane, which is visible in this case, The key to compute Z is to calibrate the internal and external parameters and to obtain the corresponding points on the projection equipment and camera image plane.

Camera calibration is a mature field, there are many methods such as the Zhang Zhengyu method, which is not analyzed here.

The matching of the projection device and the camera image plane is another key step. This is identical to the principle of binocular matching. The only difference is that the method of structured light ranging provides more auxiliary information than the pure image processing method of binocular matching when the corresponding point matching is carried out. What auxiliary information is provided to help quickly and accurately match point matching is the standard of structured light coding method.

Code character design and recognition are the key points of the design of structured light coding. In addition, because of the noise in the image itself, how the code is designed so that it can be corrected (similar to the channel Code in communication) by projecting a pre-designed pattern as a reference image (a coded light source), projecting into space, using a camera to photograph a pattern projected into space as a test image, Two images are also obtained, one is a pre-designed reference image, the other is the camera to obtain the image, the same can be used to match the image, this method is better than the binocular matching place is that the reference image is not acquired, but is a specially designed design, so the feature point is known, and more easily extracted from the test image;

More advanced is that you can use the method of encoding structured light, these designs are coded (which can be thought of as a projection image of a finite set of unique child patterns), and the patterns in each unique image window in the reference image are unique; the image of the space projection through the camera, after processing, Start looking for unique coded patterns (pattern recognition) in the captured test image. Find a pattern ID, you can directly in the reference image look-up table to find the corresponding (each design of the pattern ID in the reference image position is determined); by encoding the structured light method, There is no need for image matching, instead, it is converted into an ID that looks for the encoded unique child pattern in the test image (similar to each design of the child pattern as a small template, which is matched to find a unique child pattern or directly to encode the ID to the test image. The ID, such as the Gray code encoding of the time series, can be obtained by decoding. Pattern encoding/decoding

In conclusion: Based on the method of coded structured light, the reference image is pre-designed and consists of a number of distinct child patterns; The position of each child pattern in the reference image is predetermined (the design of the reference pattern includes the design of the child pattern and the position ID of the child pattern in the reference pattern). You can think of a lookup table that has the ID and position of each child pattern in a reference pattern:

The position of the child pattern ID in the reference pattern (or it can be an equal angle, depending on how the triangulation of the range is established)
0 (0,1)
1 (3,4)
。。。。。

This pre-designed reference pattern is projected into space through a projection device. Using the camera to shoot the reference pattern in the space, get the test image, find the ID of the unique child pattern in the test image, get a real position, check the table by ID to get its position in the reference pattern, thus complete the image matching.

Another more advanced approach is that the child patterns themselves may be the encoding that contains the location, so the direct decoding of the child pattern can get the location, even look-up table does not need.

The above is an example, through the time frame to divide the space into different regions, the test image is multiple, by stacking can get the binary encoded space position ID, here is to divide the space into 32 sub regions, each with the same coding, no distinction

The structure light coding is based on the principle of binocular ranging, but it solves the difficulty of image matching in binocular matching, so it is more effective than pure binocular matching.
2. Pseudo-Random sequence
If a sequence, on the one hand it can be predetermined, and can be repeatedly produced and replicated, on the one hand, it has a random sequence of random characteristics (that is, statistical characteristics), we call this sequence is pseudo-random sequence. So it can be said that pseudo random sequence is a certain sequence with some random characteristics. They are generated by the shift register to determine the sequence, but they have some random sequence of random characteristics. Because of the same random characteristics, it is impossible to judge the true random sequence or the pseudo random sequence from the characteristic of an already produced sequence, only according to the generation method of the sequence. Pseudo-Random sequence series has good randomness and correlation function close to white noise, and has the prior certainty and repeatability.

The so-called random-fingered correlation function is Gauss, and only autocorrelation is 1, which is close to 0.

The so-called pseudo: refers to the resulting method is determined, that is, this sequence can have a certain method to repeat the generation, and has a cyclical, rather than the real random sequence is not repeatable and not cyclical. In communication from the usual use of LFSR linear feedback shift registers to produce pseudo random sequences:

Assuming that the initial state (X4,X3,X2,X1) is 1000

Now start shifting: x1+x2 = 0, the new move is 0, then the value in the current register is 0100, output 0

Repeated: 1000,0100,0010,1001,1100,0110,1011,0101,1010,1101,1110,1111,0111,0011,0001,1000.

Output is: 111101011001000 (the first output on the right)

One-dimensional random code:

Periodicity: Theoretically the maximum period is 2^4 = 16, that is, each register is traversed by the number of 0 and 12 values, the permutation of the example of the cycle = 16, after 16 shift after the value of the register restored to the original value.
Window features: Sliding with a smaller window, the resulting sequence within the window is unique. This can be inferred from the above example, you can see that the window 4 is exactly the value of the register at some point, is definitely unique.
Folding principle: For example, a 1-dimensional pseudo-random sequence, which can be folded in any position into high dimensions such as 2-dimensional sequences is still pseudo random. For example, a sequence with a period of 3^9-1=19682 can be folded into a two-dimensional pseudo random sequence of 26x757=19682. The M-array folding method is as follows:

Multivariate domain: The number of values per register, such as only 0/1 binary, then is two domain, for images such as have r/g/b three values, then is 3 domain, and so on. The following figure is a pseudo random pattern of RGB in a 3-tuple field:

There are also two-dimensional pseudo-random codes, such as:

Random codes form a KXL matrix, window size is vxw, and each window contains a code word that appears only once. The same features of one-dimensional pseudo random codes include window characteristics, periodicity, etc.

In structured light coding, assuming that a pseudo-random sequence is known to produce methods and values, we know the following information:

Cycle size, window size, this known pseudo-random sequence is either a reference image or a coded template. Use the correct window size in the match to slide the picture in the shot, decode the pattern in the window and get a fragment of the pseudo random sequence in the window, this fragment searches for the same fragment in a known coding template, and if the distance is 0, the matching point is obtained.

It can be seen that the method of structured light coding needs to know the coding template in advance and extract the fragment information correctly in the test image. Because the pseudo random sequence has the window characteristics (the symbol within the window is unique), it is more suitable for structural light coding.

Laser speckle : The diffuse reflection of a laser on the surface of a scattering body or through a transparent scattering body (such as a glass) can be observed on a scattering surface or in a nearby light field, which is called a laser speckle (speckles).
Laser speckle is a stochastic process, which is produced by the coherent light irradiation of the random scattering body. To study it must use the method of probability statistics. Through the study of statistical methods, the strength distribution, contrast and speckle motion of speckle are recognized. The most important feature is that the speckle is highly random and varies with the distance, which means that the pattern of speckles in any two places in the same space is different. As long as the space to play such a structural light and then memory so that the entire space is marked, and then put an object into the space only need to change from the object's speckle pattern to know the exact location of the object.
Application:

The roughness of the reflecting surface is measured by the contrast of speckle;

The velocity of motion of the object is measured by the dynamic situation of speckle;

The use of speckle for optical information processing, and even the use of speckle optometry.
Lasers have great potential in the field of imaging. But "spot" problem has been puzzling people: when the traditional laser is used for imaging, due to high altitude coherence, will produce a large number of random spots or granular patterns, seriously affect the imaging effect. One way to avoid this distortion is to use an LED light source. But the problem is that for high speed imaging, the brightness of the LED light source is not enough.

structure Light : firstly, the structure light is projected onto the surface of the object, then the camera receives the structural light pattern reflected by the object's surface, because the receiving pattern will deform because of the stereo shape of the object, so it can try to calculate the space information of the object surface by the position and degree of deformation of the pattern on the camera The common structural light method is still partly based on the triangulation principle of depth calculation.
Unlike structured light, the light source of the Light coding is called "Laser speckle", which is a randomly formed diffraction spot when a laser hits a coarse object or penetrates a capillary glass. These speckles have a high degree of randomness, and will change patterns as distances vary. In other words, any two spots in the space are different. As long as the structure of this light in space, the entire space is marked, put an object into the space, just look at the speckle pattern on the object, you can know where the object is. Of course, before this to the entire space of the speckle patterns are recorded, so first to do a light source calibration.
To sum up, the difference between Light coding and traditional tof and structured light technology is as follows:
1) and the traditional tof, structured light source, laser speckle is when the laser irradiation to the coarse object or through the hair glass formed by random diffraction spots;
2 Do not need a special photosensitive chip, only the ordinary CMOS photosensitive chip;
3) Light Coding technology is not solved by the spatial geometry relation, its measuring precision is only related to the density of reference plane, and the more accurate the reference surface is. The traditional structured light method uses triangular parallax ranging, the longer the baseline length (the distance between the light source and the lens optical heart) the better. In other words, you don't have to widen the baseline to improve your accuracy. The mystery of this is the "laser speckle principle."

Disadvantages:

The laser emits a coded spot that is easily flooded with sunlight.

The laser lifetime of the Structured light scheme is short, it is difficult to meet the long time work requirement of 7*24 hours, and its long continuous work is easily damaged. Since monocular lenses and lasers need to be calibrated accurately, once damaged, it is very difficult to calibrate the two when replacing a laser, so it often leads to the whole module being replaced together.