Principle of camshift Algorithm and Its opencv implementation

 

Camshift Principle
Camshift uses the target color histogram model to convert the image into a color probability distribution chart, initialize the size and position of a search window, and adjust the position and size of the search window according to the results of the previous frame, to locate the center of the target in the current image.

There are three parts:
1 -- color projection (reverse projection ):
(1) RGB color space is sensitive to illumination brightness changes. To reduce the effect of changes on the tracking effect, the image is first converted from RGB space to HSV space. (2 ). then, we use the H component as the histogram, which represents the probability or number of pixels of different H component values. That is to say, we can find the probability or number of pixels with H component size as H, the color probability search table is obtained. (3). Replace the values of each pixel in the image with the probability pairs of their colors to obtain the color probability distribution chart. This process is called reverse projection. The color probability distribution chart is a gray image.

2 -- meanshift
The meanshift algorithm is a non-parameter method for density function gradient estimation. It finds the Extreme Value of probability distribution through iterative optimization to locate the target.
The algorithm process is as follows:
(1). Select the search window W in the color probability distribution chart.
(2). Calculate the zero degree:

Calculate the first degree:

Calculate the center of the search window:

(3). Adjust the size of the search window
The length is 1.2 s;
(4 ). move the center of the search window to the center of the center. If the moving distance is greater than the preset fixed threshold, Repeat 2) 3) 4 ), until the distance between the center and the center of the search window is smaller than the preset fixed threshold, or the number of cyclic operations reaches a certain maximum, the calculation is stopped. Proof of the convergence of meanshift can be found in the relevant Google literature.

3 -- camshift
Extend the meanshift algorithm to the continuous image sequence, namely the camshift algorithm. It performs the meanshift operation on all frames of the video and uses the result of the previous frame, that is, the size and center of the search window, as the initial value of the next meanshift algorithm's search window. After such iteration, the target tracking can be realized.
The algorithm process is as follows:
(1) initialize the search window
(2) Calculate the color probability distribution of the search window (reverse projection)
(3) run the meanshift algorithm to obtain the new size and position of the search window.
(4). reinitialize the size and position of the search window with the value (3) in the next video image, and then jump to (2) to continue.

Camshift can effectively solve the problem of Object Deformation and occlusion. It has low requirements on system resources and low time complexity, and can achieve good tracking effect in a simple background. However, when the background is complex or there are many pixel interference similar to the target color, the tracking will fail. Because it only considers the color histogram and ignores the spatial distribution of the target, you need to add a prediction algorithm for the Tracking target in this case.

Opencv Implementation of camshift
Original http://blog.csdn.net/houdy/archive/2004/11/10/175739.aspx

1 -- Back Projection
Computes the opencv code of Back Projection.
(1) Prepare an image that only contains the target to be tracked, convert the color space to the HSI space, and obtain the H component:
Iplimage * target = cvloadimage ("target.bmp",-1); // load the image
Iplimage * target_hsv = cvcreateimage (cvgetsize (target), ipl_depth_8u, 3 );
Iplimage * target_hue = cvcreateimage (cvgetsize (target), ipl_depth_8u, 3 );
Cvcvtcolor (target, target_hsv, cv_bgr2hsv); // convert to HSV space
Cvsplit (target_hsv, target_hue, null); // obtain the H component

(2) Calculate the histogram of the H component, I .e. 1D histogram:
Iplimage * h_plane = cvcreateimage (cvgetsize (target_hsv), ipl_depth_8u, 1 );
Int hist_size [] = {255}; // quantize the value of H to [0,255]
Float * ranges [] ={ {0,360}; // The value range of the H component is [0,360)
Cvhistogram * hist = cvcreatehist (1, hist_size, ranges, 1 );
Cvcalchist (& target_hue, Hist, 0, null );
Here we need to consider the value range of the H component. The value range of the H component is [0,360). The value of this value range cannot be expressed by a byte. In order to be expressed by a byte, the H value needs to be quantified appropriately. Here we can quantify the range of the H component to [0,255].

(3). Calculate Back Projection:
Iplimage * rawimage;
// Get from video frame, unsigned byte, one channel
Iplimage * result = cvcreateimage (cvgetsize (rawimage), ipl_depth_8u, 1 );
Cvcalcbackproject (& rawimage, result, hist );
(4). result is what we need.

2 -- Mean Shift Algorithm
The centroid can be calculated using the following formula:
(1). Calculate the moment of order 0 in the region
For (INT I = 0; I For (Int J = 0; j <width; j ++)
M00 + = I (I, j)

(2) moment of level 1 in the region:
For (INT I = 0; I For (Int J = 0; j <width; j ++)
{
M10 + = I * I (I, j );
M01 + = J * I (I, j );
}

(3). the mass center is:
XC = M10/m00; YC = M01/m00

In opencv, the mean shift algorithm function is provided. The prototype of the function is:
Int cvmeanshift (iplimage * imgprob, cvrect win,
Cvtermcriteria criteria, cvconnectedcomp * out );
The required parameters are:
(1). iplimage * imgprob: 2D probability distribution image, passed in;
(2) cvrect win: initial window, passed in;
(3). cvtermcriteria criteria: Criteria for stopping iteration, passed in;
(4). cvconnectedcomp * out: the query result is output.
Note: To construct a cvtermcriteria variable, three parameters are required: one is type, the other is the maximum number of iterations, and the last one represents a specific threshold. For example, you can construct criteria as follows:
Criteria = cvtermcriteria (cv_termcrit_iter | cv_termcrit_eps, 10, 0.1 ).

3 -- camshift Algorithm
The specific steps of the entire algorithm are divided into five steps:
Step 1: set the entire image as the search area.
Step 2: the size and position of the search window.
Step 3: Calculate the color probability distribution in the search window. The area size is slightly larger than that in the search window.
Step 4: Run meanshift. Obtain the new position and size of the search window.
Step 5: In the next video image, use the value obtained in step 3 to initialize the position and size of the search window. Jump to Step 3 to continue running.

Opencv code:
In opencv, there is a function that implements the camshift algorithm. The prototype of this function is:
Cvcamshift (iplimage * imgprob, cvrect win,
Cvtermcriteria criteria,
Cvconnectedcomp * Out, cvbox2d * box = 0 );
Where:
Imgprob: Color probability distribution image.
Repeated win: the initial value of search window.
Criteria: a criterion used to determine whether the search is stopped.
Out: Save the calculation result, including the location and area of the new search window.
Box: contains the smallest rectangle of the tracked object.

For more information, see:

For the opencv implementation code of the camshift algorithm with annotations, see:
Http://download.csdn.net/source/1663015