Depth Image Registration (registration) principle

In machine vision, the depth image (depth image) produced by a 3D camera typically requires registration (registration) to generate a registration depth image (registed depth image). In fact, the purpose of registration is to make the depth map and color graph coincident, that is, the image coordinate system of the depth image is converted to the color image coordinate system. Let's describe the process of its derivation.

To describe the convenience, first make some simple assumptions. The left camera of the 3D camera is an infrared camera (that is, a depth camera, an IR camera), and a color camera (the right camera). Now the mainstream 3D camera is such a layout, such as Xtion,kiinect,orbbict. As shown in the following:

The pixels of the known color image are represented by \ ((U_{r}, V_{r}, z_{r}), \ (U_{r}, V_{r}, z_{r}\) respectively, representing the horizontal axis of the color image, the depth value in the ordinate and camera coordinates (the value in the z direction, the distance from the two point); , the pixels of the color image are \ ((U_{l}, V_{l}, z_{l}), \ (U_{l}, v_{l}, z_{l}\) respectively, representing the horizontal axis of the color image, the depth value in the ordinate and camera coordinates (the value in the z direction, the distance not two points). Note that in order to facilitate the expression, the r,l of the subscript in this paper represents right,left meaning respectively. Then the registration process is to find the transformation matrix in the following formula \ (w^{'}\):

\ (\begin{bmatrix}
u_{r}\\
v_{r}\\
1
\end{bmatrix}
=w^{'}
\begin{bmatrix}
u_{l}\\
v_{l}\\
1
\end{bmatrix}\)

In fact, the transformation matrix \ (w^{'}\) is not really a matrix to solve

Depth Image Registration (registration) principle