SFM: Recovering structures from motion

Reprint Address: http://blog.csdn.net/aichipmunk/article/details/48132109

SFM Introduction

The entire SFM, called Structure from Motion, is a common method for three-dimensional reconstruction by determining the spatial and geometrical relationship of a target through the movement of the camera. The biggest difference between this 3D camera and Kinect is that it only needs a normal RGB camera, so it's cheaper and less constrained by the environment and can be used indoors and outdoors. However, the SFM needs complex theory and algorithm to support, in the precision and speed are still to be improved, so there are not many mature business applications.
This series introduces the fundamentals and algorithms in SFM and implements a simple SFM system with OPENCV. Small hole camera model

In computer vision, the most commonly used camera model is the small hole model (keyhole imaging model), which simplifies the lens set of the camera into a small hole, and the light is imaged through a small hole in the image on the rear of the hole, as shown in the figure below.

From the above figure, the small hole model is inverted image, in order to express and research convenience, we often will like the surface of the small hole before, and the distance to the hole is still the focal length F, such model and the original hole model is equivalent, but is just like, in line with the intuitive feelings of people. In this case, the pores are often referred to as the Light Center (Optical Center).

The small hole model is an ideal camera model, without considering the problems of field curvature and distortion in the actual camera. In practical use, these problems can be solved by introducing distortion parameters in the process of calibration, so the hole model is still the most widely used camera model. coordinate system

In order to study SFM with mathematics, we need a coordinate system. There are two main types of coordinate systems in SFM, one is camera coordinate system and the other is world coordinate system. In this series, the coordinate system is the right-handed coordinate system.
The camera coordinate system takes the camera's center of light (the small hole) as the origin, the x axis is horizontal, the Y axis is vertical, and the z axis points to the direction the camera is looking at.
The origin of the world coordinate system can be arbitrarily selected, regardless of the camera's specific position.
Internal Reference Matrix

There is a little p in the space, and if the world coordinate system is coincident with the camera coordinate system, the point is in space with coordinates (X, Y, z), where Z is the vertical distance from the point to the camera's center of light. What is the relationship between (x, Y, Z) and (x, y) the pixel coordinates (x, y) of this point on the image surface as point P?

This is a simple, similar triangular relationship that gets the
X=fxz,   y=fyz
, but the image's pixel coordinate system origin is in the upper-left corner, and the upper formula assumes the origin is in the center of the image, In order to handle this offset, the corresponding pixel coordinate of the optical center on the image is (cx,cy), then
X=fxz+cx,   y=fyz+cy
represents the above relationship as a matrix, with
z⎡⎣⎢xy1⎤⎦⎥= ⎡⎣⎢f000f0cxcy1⎤⎦⎥⎡⎣⎢xyz⎤⎦⎥
where the matrix
k=⎡⎣⎢f000f