Paper notes: Optical Flow estimation using a Spatial Pyramid Network

　　Optical Flow estimation using a Spatial Pyramid Network

SpyNet

In this paper, the classical spatial-pyramid formulation and deep learning method are combined to calculate the optical flow with a kind of coarse to fine approach. This estiamates large motions in a coarse to fine approach by warping one image of a pair at each pyramid level by the cur Rent flow estimate and compute an update to the flow.

Instead of minimizing the objective function in traditional methods, we use CNN to update each layer of flow. Compared with flownet, this method does not need to deal with large motions; these are already processed in pyramid. The main advantages of this approach are:

1. Our Spatial Pyramid Network are much simpler and 96% smaller than flownet in terms of model parameters.

2. since the Flow at each pyramid level is small (< 1 pixel), a convolutional  Approach applied to pairs of warped images are appropriate.  

3. Unlike Flownet, the learned convolution filters appear similar to classical spatio-temporal filters, giving Insight Into the method and how to improve it.

The main problems existing methods are:

if a convolutional window in one image does not overlap with related image pixels at the NEX T time instant, no meaningful temporal filter can be learned. 

There are two key issues to solve: 1. The problem of long-term dependence; 2. Detailed, sub-pixel, optical flow and precise motion boundaries. Flownet is trying to solve these two problems in a network, and the method is to use CNN to solve the second problem, using the existing method to solve the first problem.

　　

Approach:

In this paper, the coarse to fine method is used to solve the problem of large motion in the way of spatial pyramid.

　　

　　

　　

　　

　　

Paper notes: Optical Flow estimation using a Spatial Pyramid Network