Optical flow algorithm: The principle of image transformation based on optical flow (monocular) or parallax (binocular)

Take the optical flow as an example.

In order to obtain a more accurate optical flow value, we usually get the J-Shift (-U,-V) WARPJ by setting the approximate optical flow of the previous frame I relative to the next frame J (u,v). Then, I calculate the optical flow (DU,DV) relative to the WARPJ. Thus, a more precise optical flow (U+DU, V+DV) is obtained. This process can be repeated to get more accurate optical flow, which is called the outer loop process in the computational framework of the optical flow.

It is important to note that the outer loop is usually executed only once, because it is time consuming to transform J to WARPJ. So we assign more accurate optical flow calculations to the inner loop process. Here, we are not discussing the inner loop process for the moment.

So how do you change J to WARPJ?

The conversion of J to WARPJ is (-u,-V), which means that the WARPJ to J is (U,V). Then, the grayscale value at WARPJ (x, y) should be equal to the gray value at J (X+u, Y+v). So, we just need to find the gray value of J (X+u, Y+v) and assign it to WARPJ (x, y). Since (X+u, y+v) is usually a non-integer coordinate, interpolation is required to calculate it.

Optical flow algorithm: The principle of image transformation based on optical flow (monocular) or parallax (binocular)