[Algorithm analysis] Lucas-kanade Optical Flow Algorithm

By gnuhpc@gmail.com

Introduction: in computer vision, the Lucas-kanade optical flow algorithm is a two-frame differential optical flow estimation algorithm. It was proposed by Bruce D. Lucas and Takeo kanade.

Concept of Optical Flow: (optical flow or optic flow)
It is a motion pattern that refers to an object, surface, and edge from an angle of view to an observer (such as eyes, cameras, etc) and the background. Optical flow technology, such as motion detection and image segmentation, time collision, motion compensation coding, and three-dimensional parallax, all use this edge or surface motion technology.

The movement of two-dimensional images is a projection of three-dimensional objects moving on the image plane relative to the observer.
Ordered images can estimate the instantaneous image rate or discrete image transfer of two-dimensional images.

Optical Flow Algorithm:
It evaluates the deformation between two images, and its basic assumption is the conservation of body elements and image pixels. It assumes that the color of an object does not change significantly between the two frames. Based on this idea, we can obtain the image constraint equation. Different Optical Flow algorithms solve the optical flow problem where different additional conditions are assumed.

Lucas-kanade algorithm:
This algorithm is the most common and popular. It calculates the movement of two frames between T and T + delta T at each pixel position.
Because it is based on the tyleseries of the image signal, this method is called difference, which is to use partial derivative for spatial and temporal coordinates.
The image constraint equation can be writtenI(X,Y,Z,T) =I(X+ DeltaX,Y+ DeltaY,Z+ DeltaZ,T+ DeltaT)
I (X, Y, Z, T)
Is the body element at the position (x, y, z.
Let's assume that the movement is small enough. Then we can use the Taylor Formula for the image constraint equation to obtain:

H.o. T. indicates a higher order, which can be ignored when moving small enough. From this equation, we can obtain:

Or

We get:

V
X
,V
Y
,V
Z
These are the components of X, Y, Z in the optical flow vector of I (X, Y, Z, t.
, And
The image is in (X,Y,Z,T) This is a difference in the corresponding direction.
.
So

I
X
V
X
+I
Y
V
Y
+I
Z
V
Z
= −I
T.

Write:

This equation has three unknown numbers, which cannot be solved yet, that is, the aperture problem of the optical flow algorithm. To find the optical flow vector, you need another solution. The Lucas-kanade algorithm is a non-iterative algorithm:

Assume that the stream (VX, Vy, VZ) is a constant in a small window with a size of M * m (M> 1), then from pixel 1...N,N=M
3, we can obtain the following equations:

There are three unknown but more than three equations. This system of equations is naturally a hyper-definite equation. That is to say, there is redundancy in the system of equations. The equations can be expressed:

Note:

In order to solve this super-Fixed problem, we adopt the least square method:

Or

Get:

The sum is from 1 to n.

This means that the search for Optical flows can be obtained by accumulating the four-dimensional image derivatives respectively. We also need a weight function.W (I, j, k)
,
To highlight the coordinates of the center of the window. Gaussian Functions are very suitable for doing this,

The disadvantage of this algorithm is that it cannot generate a high-density flow. For example, the stream information in the tiny movement of the moving edge and the black-large homogeneous area will quickly fade away. Its advantage lies in its robustness to noise.

Supplement: the implementation in opencv seems complicated, but it is not clear yet. Among them, the LK classical algorithm is also an iterative method, which is used to calculate linear equations by Gaussian iteration.

References: Lucas B and kanade T. an iterative Image Registration Technique with an application to stereo vision. proc. of 7th International Joint Conference on Artificial Intelligence (ijcai), pp.674-679. published in 81 years.

Opencv implements an algorithm published in 84 years,

References: Bruce D. Lucas, "Generalized image matching by the method of differences," doctoral dissertation, tech. Report, Robotics Institute, Carnegie Mellon University, July, 1984

This paper is charged and has never been paid back.