Opencv learning notes (11) Target Tracking

Original article, reproduced Please note: http://blog.csdn.net/crzy_sparrow/article/details/7414851

If the camera is fixed, we can consider that the scene (background) remains unchanged in most cases, and only the foreground (the target to be tracked) moves, so that we can establish a background model. By comparing the current frame and background model, you can easily track the target motion. Here, the easiest way to compare is to subtract the background model from the current frame. For example, the following function in opencv2.3.1 can calculate the absolute value of the difference between the current frame and the background.

CV: absdiff (backgroundimage, currentimage, foreground );

The easiest thing to think about when a background model is created is to select an image with no target as the background model. However, in many cases, the background usually has a target moving, for example, traffic monitoring may keep rolling around, and the program will not know which image has a target and which image has no target if it has no experience. Okay, I admit that I am confused. modeling is built by us, and we can choose to create images based on our perception, the program only detects the target based on the model we have created. However, as I have said, it may be difficult to obtain images without targets, and you certainly think it is difficult to use only one image without targets as a background model, in other words, you have to calculate an average value for multiple background images. As a result, we can capture a relatively small number of video segments (also called image sequences) at the target ...), Then calculate the mean value of the frame and use the mean image as the background model. At this time, the background model should always be reliable.

The problem comes again. My camera is fixed, but the weather is always a function of time. The background of the day and night must be different. Besides those buildings, mountains, the positions of objects in the background may change with time, or move or disappear. For example, a billboard in the background is blown down by the wind, A tree was cut down... At this time, we should dynamically update the background model. Modifying the previous mean background model can also solve this problem: it is to reconstruct a background model at intervals. In this way, you will encounter several problems: 1) Before modeling, a large number of images will occupy a large amount of storage space; 2) it will take a long time to create a model, so the target cannot be tracked; 3) selecting several frames for image modeling is a problem. 2) It is not difficult to solve the problem. First, create an average model, record the mean model and each frame of image, and then each frame of image will be taken, seek its foreground target, and delete the original first frame of image, insert the background image of the current frame to the end of the modeling image, and update the background model. Solution 1): create a model first, and no longer store the modeling image. Each frame is sent, and the foreground target is calculated, use the remaining background images to directly update the original model (Model * n + I)/(n + 1) (model image, n is the number of modeling images ). However, there is a drawback: with the passage of time, the background has changed a lot, that is, the last frame in the modeling image is very different from the first frame, the last frame is closer to the current background. Therefore, we can assign different weights to the modeled images in order to increase the weights of the subsequent images. This is the target tracking algorithm.

If the model is the current background model and the cur is the image after the background is removed from the current frame, the new model is:

Modelnew = (1-A) * model + A * cur; A is the learning rate. Over time, the weights of the previous modeling images become smaller and smaller.

Opencv2.3.1

Void accumulateweighted (inputarray SRC, inputoutputarray DST, double alpha, inputarray mask = noarray ())
Function:

Its parameters are described as follows:

Parameters
Src-input image as 1-or 3-channel, 8-bit or 32-bit floating point.
DST-accumulator image with the same number of channels as input image, 32-bit or 64-bit floating-point.
Alpha-weight of the input image.
Mask-optional operation mask.
The function calculates the weighted sum of the input image SRC and the accumulator DST so that DST becomes
Running average of a frame sequence:
DST (x, y) hour (1 −alpha) · DST (x, y) + Alpha · SRC (x, y) If mask (x, y) = 0

The opencv code for achieving the target tracking using this method is as follows. Here I use the first frame of the image as the initial background model:

// Frame processing base class frameprocessor {public: Virtual void process (MAT & input, mat & ouput) = 0 ;}; class bgfgsegmentor: Public frameprocessor {mat gray; // current frame grayscale mat background; // background image, in the format of 32-bit floating point mat backimage; // cv_8u format background image mat foreground; // foreground image double learningrate; // learning rate int threshold; // threshold, filter out disturbance public: bgfgsegmentor (): threshold (30), learningrate (0.6) {} // frame processing function void process (MAT & frame, mat & output) {// converts it to a grayscale image cvtcolor (frame, gray, cv_bgr2gray); If (background. empty () // The first gray frame. convertize (background, cv_32f); // convert the background to cv_8u format to obtain the absolute value background that is different from the current frame. convertize (backimage, cv_8u); // calculate the difference between the current frame and the background (absdiff (backimage, gray, foreground); // filter out the disturbance points CV with little difference between the foreground and the background:: threshold (foreground, output, threshold, 255, thresh_binary_inv); // update the background. output is used as the mask accumulateweighted (Gray, background, learningrate, output );}};

The result is as follows:

The video processing class and main function used are as follows:

Class videoprocessor {PRIVATE: videocapture caputure; // write the Video Stream object videowriter writer; // output file name string outputfile; int currentindex; int digits; string extension; frameprocessor * frameprocessor; // image processing function pointer void (* process) (MAT &, mat &); bool calender; string windownameinput; string windownameoutput; // delay int delay; long fnumber; // frametostop long frametostop; // the stop sign bool stop; // The image sequence is used as the input video stream vector <strin G> images; // iterator public: videoprocessor (): calender (true), delay (0), fnumber (0), stop (false), digits (0 ), frametostop (-1) {}// sets the image processing function void setframeprocessor (void (* process) (MAT &, mat &) {frameprocessor = 0; this-> process = process; callprocess () ;}// open the video bool setinput (string filename) {fnumber = 0; // if enabled, release and re-open caputure. release (); Return caputure. open (filename);} // sets the void displayinput (Str Ing wn) {windownameinput = wn; namedwindow (windownameinput);} // set the void displayoutput (string wn) {windownameoutput = wn; namedwindow (windownameoutput );} // destroy window void dontdisplay () {destroywindow (descriwnameinput); destroywindow (descriwnameoutput); descriwnameinput. clear (); windownameoutput. clear () ;}// start void run () {mat frame; MAT output; If (! Isopened () return; stop = false; while (! Isstopped () {// read the next frame if (! Readnextframe (FRAME) break; If (windownameinput. Length ()! = 0) imshow (Fig input, frame); // process the frame if (calender) {If (process) process (frame, output); else if (frameprocessor) frameprocessor-> process (frame, output);} else {output = frame;} If (outputfile. length () {cvtcolor (output, output, cv_gray2bgr); writenextframe (output);} If (windownameoutput. length ()! = 0) imshow (windownameoutput, output); // press the button to pause. Press the next button to continue. If (delay> = 0 & waitkey (Delay)> = 0) waitkey (0 ); // when the specified sequence key is reached, exit if (frametostop> = 0 & getframenumber () = frametostop) stopit () ;}// the sequence key is set to void stopit () {stop = true;} // query the pause flag bool isstopped () {return stop;} // return the opening flag bool isopened () {return caputure. isopened () |! Images. empty ();} // sets the delay void setdelay (INT d) {delay = D;} // reads the next frame bool readnextframe (MAT & frame) {If (images. size () = 0) return caputure. read (FRAME); else {If (itimg! = Images. End () {frame = imread (* itimg); itimg ++; return frame. Data? 1:0;} else return false;} void callprocess () {calender = true;} void dontcallprocess () {calender = false;} // you can specify void stopatframeno (long frame) to stop a frame) {frametostop = frame;} // obtain the position of the current frame long getframenumber () {long fnumber = static_cast <long> (caputure. get (cv_cap_prop_pos_frames); Return fnumber;} // get the frame size getframesize () {If (images. size () = 0) {// obtain the frame size from the video stream int W = static_cast <int> (caputure. g Et (cv_cap_prop_frame_width); int H = static_cast <int> (caputure. get (cv_cap_prop_frame_height); Return size (W, H);} else {// obtain the frame size from the image CV: mat TMP = CV :: imread (images [0]); Return (TMP. data )? (TMP. size () :( size (0, 0) ;}// get frame rate double getframerate () {return caputure. get (cv_cap_prop_fps);} vector <string >:: const_iterator itimg; bool setinput (const vector <string> & IMGs) {fnumber = 0; caputure. release (); images = IMGs; itimg = images. begin (); Return true;} void setframeprocessor (frameprocessor * frameprocessor) {process = 0; this-> frameprocessor = frameprocessor; callprocess ();} // get Encoding type int getcodec (char codec [4]) {If (images. Size ()! = 0) Return-1; Union {// data structure 4-Char int value; char code [4];} returned; // obtain the encoding value returned. value = static_cast <int> (caputure. get (cv_cap_prop_fourcc); // get the 4 characters codec [0] = returned. code [0]; codec [1] = returned. code [1]; codec [2] = returned. code [2]; codec [3] = returned. code [3]; return returned. value;} bool setoutput (const string & filename, int codec = 0, double framerate = 0.0, bool iscolor = true) {// set the file name outputfile = filename; // clear extension. clear (); // set the frame rate if (framerate = 0.0) {framerate = getframerate () ;}// obtain the encoding method of the input original video. Char C [4]; if (codec = 0) {codec = getcodec (c);} return writer. open (outputfile, codec, framerate, getframesize (), iscolor);} // The output video frame is sent to filemetriccurrentindex.ext, for example, filename001.jpg bool setoutput (const string & filename, // path const string & Ext, // extension int numberofdigits = 3, // number of digits int startindex = 0) {// start index if (numberofdigits <0) return false; outputfile = filename; Extension = ext; digits = numberofdigits; currentindex = startindex; return true;} // write down a void writenextframe (MAT & frame) {// if the extension is not empty, if (extension. length () {stringstream SS; SS <outputfile <setfill ('0') <SETW (digits) <currentindex ++ <extension; imwrite (ss. STR (), frame);} // conversely, write to the video file else {writer. write (FRAME) ;}}; // frame processing function: Specifies the void of the video edge detection function (CV: mat & IMG, CV: mat & out) {// grayscale transform if (IMG. channels () = 3) cvtcolor (IMG, out, cv_bgr2gray); // you can find the edge by using the "out", "out", and "100,200". // The color is reversed, threshold (Out, out, 128,255, CV: thresh_binary_inv);} int main (INT argc, char * argv []) {videoprocessor processor; // featuretracker tracker; bgfgsegmentor tracker; // open the input video processor. setinput ("bike. avi "); processor. displayinput ("current frame"); processor. displayoutput ("output frame"); // sets the processor latency of each frame. setdelay (1000. /processor. getframerate (); // sets the frame processing function, which can be any processor. setframeprocessor (& tracker); // processor. setoutput (". /bikeout. avi "); // processor. setoutput ("bikeout ",". jpg "); processor. run (); Return 0 ;}