Learning notes of opencv -- camshift Algorithm for Moving Object Tracking

|Font Size Subscription

Introduction
CamshiftAlgorithmThat is, the "continuously apative mean-shift" algorithm is a motion tracking algorithm. It is used to track the color information of moving objects in video images. I divide this algorithm into three parts for ease of understanding:

Back Projection calculation.

Mean Shift Algorithm

Camshift Algorithm

1 Back Projection Calculation
The steps for calculating back projection are as follows:

1. Calculate the color histogram of the tracked target. In various color spaces, only the H component in the HSI space (or a color space similar to the HSI space) can represent color information. Therefore, in the specific calculation process, the values of other color spaces are first converted to the HSI space, and then the H component is used for 1d histogram calculation.

2. Convert the original image into a color Probability Distribution Image Based on the obtained color histogram. This process is called "Back Projection ".

The histogram function in opencv contains the back projection function. The function prototype is:

Void cvcalcbackproject (iplimage ** IMG, cvarr ** backproject, const cvhistogram * hist );

There are three parameters passed to this function:

1. iplimage ** IMG: stores the original image and inputs it.

2. cvarr ** backproject: stores the Back Projection result and outputs it.

3. cvhistogram * hist: stores the histogram and inputs

The following section describes the opencv for Back Projection calculation.Code.

1. Prepare an image that only contains the target to be tracked, convert the color space to the HSI space, and obtain the H component:

Iplimage * target = cvloadimage ("target.bmp",-1); // load the image

Iplimage * target_hsv = cvcreateimage (cvgetsize (target), ipl_depth_8u, 3 );

Iplimage * target_hue = cvcreateimage (cvgetsize (target), ipl_depth_8u, 3 );

Cvcvtcolor (target, target_hsv, cv_bgr2hsv); // convert to HSV space

Cvsplit (target_hsv, target_hue, null); // obtain the H component

2. Calculate the histogram of H component, I .e. 1D histogram:

Iplimage * h_plane = cvcreateimage (cvgetsize (target_hsv), ipl_depth_8u, 1 );

Int hist_size [] = {255}; // quantize the value of H to [0,255]

Float * ranges [] ={ {0,360}; // The value range of the H component is [0,360)

Cvhistogram * hist = cvcreatehist (1, hist_size, ranges, 1 );

Cvcalchist (& target_hue, Hist, 0, null );

Here we need to consider the value range of the H component. The value range of the H component is [0,360). The value of this value range cannot be expressed by a byte. In order to be expressed by a byte, the H value needs to be quantified appropriately. Here we can quantify the range of H to [0,255].

4. Calculate Back Projection:

Iplimage * rawimage;

//----------------------------------------------

// Get from video frame, unsigned byte, onE Channel

//----------------------------------------------

Iplimage * result = cvcreateimage (cvgetsize (rawimage), ipl_depth_8u, 1 );

Cvcalcbackproject (& rawimage, result, hist );

5. Result: The result is what we need.

2) Mean Shift Algorithm

Here we come to the camshift algorithm, the second part of opencv implementation. This time we will focus on the Mean Shift algorithm.

Before discussing the Mean Shift algorithm, we first discuss how to calculate the center of gravity (Mass Center) of a region in a 2D probability distribution image. The center of gravity can be calculated using the following formula:

1. Calculate the moment of order 0 in the region

For (INT I = 0; I

For (Int J = 0; j <width; j ++)

M00 + = I (I, j)

2. Moment of level 1 in the region:

For (INT I = 0; I

For (Int J = 0; j <width; j ++)

{

M10 + = I * I (I, j );

M01 + = J * I (I, j );

}

3. The mass center is:

XC = M10/m00; YC = M01/m00

Next, we will discuss the specific steps of the Mean Shift algorithm. The Mean Shift algorithm can be divided into the following four steps:

1. Select the window size and initial position.

2. Calculate the Mass Center in the window.

3. Adjust the center of the window to the Mass Center.

4. Repeat 2 and 3 until the window center is "converged", that is, the distance between each window movement is smaller than a certain threshold.

In opencv, the mean shift algorithm function is provided. The prototype of the function is:

Int cvmeanshift (iplimage * imgprob, cvrect win,

Cvtermcriteria criteria, cvconnectedcomp * out );

The required parameters are:

1. iplimage * imgprob: 2D probability distribution image, passed in;

2. cvrect win: initial window, passed in;

3. cvtermcriteria criteria: Criteria for stopping iteration, passed in;

4. cvconnectedcomp * out: the query result is output.

(Note: To construct a cvtermcriteria variable, three parameters are required. One is type, the other is the maximum number of iterations, and the last one represents a specific threshold. For example, you can construct criteria: Criteria = cvtermcriteria (cv_termcrit_iter | cv_termcrit_eps, 10, 0.1 ).)

Returned parameters:

1. INT: number of iterations.

Implementation Code: temporarily missing

3) camshift Algorithm
1. Principle

After learning about the meanshift algorithm, we extend the meanshift algorithm to the continuous image sequence (generally the video image sequence), thus forming the camshift algorithm. The camshift algorithm is called "continuously apapaptive mean-shift". Its basic idea is to perform the meanshift operation on all frames of a video image, the result of the previous frame (that is, the center and size of the search window) is used as the initial value of the search window of the next frame of the meanshift algorithm. In this way, the target tracking can be realized through iteration. The specific steps of the entire algorithm are divided into five steps:

Step 1: set the entire image as the search area.

Step 2: the size and position of the search window.

Step 3: Calculate the color probability distribution in the search window. The area size is slightly larger than that in the search window.

Step 4: Run meanshift. Obtain the new position and size of the search window.

Step 5: In the next video image, use the value obtained in step 3 to initialize the position and size of the search window. Jump to Step 3 to continue running.

2. Implementation

In opencv, there is a function that implements the camshift algorithm. The prototype of this function is:

Cvcamshift (iplimage * imgprob, cvrect win,

Cvtermcriteria criteria,

Cvconnectedcomp * Out, cvbox2d * box = 0 );

Where:

Imgprob: Color probability distribution image.

Repeated win: the initial value of search window.

Criteria: a criterion used to determine whether the search is stopped.

Out: Save the calculation result, including the location and area of the new search window.

Box: contains the smallest rectangle of the tracked object.

Note:

1. in the directory of opencv 4.0 Beta, there is an example of camshift. Unfortunately, the target tracking in this example is semi-automated, that is, you need to manually select a target. I am trying to implement fully-automated target tracking. I hope you can communicate with us in this regard.