Understanding of meanshift tracking (implemented in opencv)

Reproduced from: http://lijunjie0704.blog.163.com/blog/static/151148274201051443615780? Fromdm & fromsearch & isfromsearchengine = Yes

The idea of the meanshift algorithm is actually very simple: the gradient climb of probability density is used to find the local optimum. All it has to do is input a range in the image, and then iterate continuously (toward the center of gravity) until it meets your requirements. But how is it used for image tracking? This is why I have been confused since I learned meanshift. There is no reasonable explanation on the Internet. After several days of thinking and understanding of reverse projection, I have a general understanding of its principles.

In opencv, meanshift is actually very simple. Input an image (imgprob), then input a circle win to start iteration, and an iteration condition (criteria ), the output is the position (COMP) of iteration completion ).

This is a function prototype:

Int cvmeanshift (const void * imgprob, cvrect win,Cvtermcriteria criteria, cvconnectedcomp * comp)

But when it is used for tracking, the input image must be a reverse projection image.

Why is reverse projection necessary? First, we need to understand what a reverse projection chart is.

A simple understanding is actually a probability density chart. During reverse projection, the input is a histogram of the target image (which can also be considered as the target image), and the input is the current image as the full graph to be tracked, the output size is as large as that of the full image. The pixels represent a probability that the point on the full image is part of the target image. If the point is brighter, it indicates that the point belongs to an object with a higher probability. Now we understand that this is a probability chart. When using meanshift for tracking, it is no wonder that the input is such an image.

 

Semi-automated tracking: Enter the video, use a paint brush to circle the target to be tracked, and then track the object.

Anyone who has used opencv knows that this is actually the working process of camshiftdemo.

Step 1: select an object and record the box and object you entered.

Step 2: Obtain the reverse projection of objects in the video.

Step 3: Perform the meanshift iteration Based on the reverse projection chart and the input box. Because it moves to the center of gravity, that is, to the place with a high probability in the reverse projection chart, it will always move to the target.

Step 4: Use the box output from the previous frame to iterate the next frame.

 

 

Automatic Tracking: input video to track moving objects.

Step 1: Use the motion detection algorithm to separate the moving object from the background.

Step 2: extract the contour of a moving object and obtain the motion image information from the source image.

Step 3: Perform reverse projection on this information to obtain the reverse projection chart.

Step 4: perform a meanshift iteration Based on the reverse projection chart and the contour of the object (that is, the input box), because it moves to the center of gravity, that is, to move to a place with a high probability in the reverse projection chart, so it will always move to the object.

Step 5: Use the box output from the previous frame to iterate the next frame.

 

Conclusion: the most important thing to use meanshift for tracking is to grasp the input image, that is, to make its iteration more and more iterative to the target. This kind of image is not necessarily a reverse projection image, as long as it is a map that reflects the probability of each pixel in the current image containing the target. In fact, a reverse projection image is just like this.