Compressive tracking -- CT tracking

 

Thanks to Kaihua Zhang of The Hong Kong Polytechnic University for his paper: Real-Time compressive tracking on eccv 2012. Here is his introduction:

A simple and efficient tracking algorithm based on compression sensing. First, the multi-scale image features are reduced by using random moment sensing that meets the compression perception rip conditions, and then the features after dimensionality reduction are classified by a simple Naive Bayes classifier. The tracking algorithm is very simple, but the experimental results are very robust and the speed can reach 40 frames/second. For detailed analysis, see related articles.

Link

Point-free download code: http://download.csdn.net/detail/sangni007/5297374

1. Description: Compute Haar features (templates)

void CompressiveTracker::HaarFeature(Rect& _objectBox, int _numFeature)

Take the _ numfeature dimension feature in the rect. (the width and height of the rect are the same as that of _ objectbox, which is irrelevant to _ objectbox. X _ objectbox. Y)
Each dimension feature is represented by several rect values and exists in the vector <rect> feature (_ numfeature, vector <rect>,

Weight Vector <vector <float> featuresweight (_ numfeature, vector <float> ())

 

2. Description: Compute the coordinate of positive and negative sample image templates

void CompressiveTracker::sampleRect(Mat& _image, Rect& _objectBox, float _rInner, float _rOuter, int _maxSampleNum, vector<Rect>& _sampleBox)

Randomly sprinkling several rect records to record coordinate rect (x, y)

Ensure that the DIST of samplerect (x, y) to _ objectbox (x, y) meets the following requirements: _ router * _ router <Dist <_ rinner * _ rinner

Sample: 1: positive: Dist small; 2: negative: Dist large:

Saved _ samplebox

void CompressiveTracker::sampleRect(Mat& _image, Rect& _objectBox, float _srw, vector<Rect>& _sampleBox)

This samplerect overload function is used to predict the location

As long as the condition is met: Dist <_ rinner * _ rinner

During initialization of the first frame, this samplerect overload function is not run. Only when tracking is performed, it is run first and then the positive and negative sample functions are updated;

 

3. compute the features of samples

void CompressiveTracker::getFeatureValue(Mat& _imageIntegral, vector<Rect>& _sampleBox, Mat& _sampleFeatureValue)

Calculate the sum value of the integral image of the corresponding rect, and store it to mat & _ samplefeaturevalue (I .e. the feature value)

 

4. // update the mean and variance of the Gaussian Classifier

void CompressiveTracker::classifierUpdate(Mat& _sampleFeatureValue, vector<float>& _mu, vector<float>& _sigma, float _learnRate)

Calculate the expectation and standard deviation of each samplebox in mat & _ samplefeaturevalue of each integral matrix in the previous step.
And update (the formula in this article is used for calculation [6])

 

5. // compute the ratio Classifier

void CompressiveTracker::radioClassifier(vector<float>& _muPos, vector<float>& _sigmaPos, vector<float>& _muNeg, vector<float>& _sigmaNeg, Mat& _sampleFeatureValue, float& _radioMax, int& _radioMaxIndex)

Formula for using Naive Bayes classification (Gaussian Model) [4]

Calculate the Bayes value of all samples and store the maximum value with radiomax, that is, the predicted position.

 

Discussion

The most successful part of this article is that it is simple and efficient, and it can definitely achieve the real-time tracking effect. The particle filter in the previous experiment is too slow, but there are many problems that need to be improved. (Personal opinion and summary from netizens. Please click it ~)

1. scale problems, only single-scale operations can be performed, and the tracking box size remains unchanged when the object is far away;

2. Compression sensing can be improved. For example, features pool is used to select the optimal feature like random forest;

3. The new sample is biased towards the current one, so it is easy to forget the samples that have been learned before. Once the deviation occurs, the samples will be farther and farther away;

4. I recently studied tracking by detection and read compressive tracking. It seems very innovative. This simple method can achieve better results. The experiment part of this paper is similar to MIL tracking, the weak mil classifier is also assumed to use Gaussian distribution, but the result of my experiment is that the Haar like feature is not well subject to Gaussian distribution, especially the negative sample, therefore, this representation cannot be captured when the characterization of objects changes greatly. In addition, there is a learning parameter in it.
Lambda, the setting of this parameter is also an experience value. In my experience, it is generally biased towards the current new sample, so it is easy to forget the samples that have been learned before, the random forest fern of TLD records the number of positive and negative samples to calculate posterior. This method is better than long time tracking, and TLD performs well in every detail, its update model also has the theoretical support of dynamic systems. These are my personal opinions and I hope to have more communication. -- Old bear

5. If you add trick to some tracking algorithms, you can also perform long time tracking. The trick of TLD uses the information of the first frame. In fact, this information is often not very good. This trick was used many years ago, not by TLD. In addition, the tld pami article does not mention the trick, cvpr10 said. CT is a simple method. Its focus is not to explain the effect of tracking, but to explain the problem in theory-zhkhua

//---------------------------------------------------class CompressiveTracker{public:CompressiveTracker(void);~CompressiveTracker(void);private:int featureMinNumRect;int featureMaxNumRect;int featureNum;vector<vector<Rect>> features;vector<vector<float>> featuresWeight;int rOuterPositive;vector<Rect> samplePositiveBox;vector<Rect> sampleNegativeBox;int rSearchWindow;Mat imageIntegral;Mat samplePositiveFeatureValue;Mat sampleNegativeFeatureValue;vector<float> muPositive;vector<float> sigmaPositive;vector<float> muNegative;vector<float> sigmaNegative;float learnRate;vector<Rect> detectBox;Mat detectFeatureValue;RNG rng;private:void HaarFeature(Rect& _objectBox, int _numFeature);void sampleRect(Mat& _image, Rect& _objectBox, float _rInner, float _rOuter, int _maxSampleNum, vector<Rect>& _sampleBox);void sampleRect(Mat& _image, Rect& _objectBox, float _srw, vector<Rect>& _sampleBox);void getFeatureValue(Mat& _imageIntegral, vector<Rect>& _sampleBox, Mat& _sampleFeatureValue);void classifierUpdate(Mat& _sampleFeatureValue, vector<float>& _mu, vector<float>& _sigma, float _learnRate);void radioClassifier(vector<float>& _muPos, vector<float>& _sigmaPos, vector<float>& _muNeg, vector<float>& _sigmaNeg,Mat& _sampleFeatureValue, float& _radioMax, int& _radioMaxIndex);public:void processFrame(Mat& _frame, Rect& _objectBox);void init(Mat& _frame, Rect& _objectBox);};

 

 

# Include "compressivetracker. H "# include <math. h ># include <iostream> using namespace CV; using namespace STD; // compressivetracker: compressivetracker (void) {featureminnumrect = 2; featuremaxnumrect = 4; // Number of rectangle from 2 to 4 featurenum = 50; // Number of all weaker classifiers, I. e, feature poolrouterpositive = 4; // radical scope of positive sampl Esrsearchwindow = 25; // size of search windowmupositive = vector <float> (featurenum, 0.0f); munegative = vector <float> (featurenum, 0.0f ); sigmapositive = vector <float> (featurenum, 1.0f); sigmanegative = vector <float> (featurenum, 1.0f); learnrate = 0.85f; // learning rate parameter} compressivetracker ::~ Compressivetracker (void) {} void compressivetracker: haarfeature (rect & _ objectbox, int _ numfeature)/* Description: Compute Haar features arguments:-_ objectbox: [x y width height] object rectangle-_ numfeature: Total number of features. the default is 50. */{features = vector <rect> (_ numfeature, vector <rect> (); featuresweight = vector <float> (_ numfeature, vector <float> (); int numrect; rect recttemp; float weighttemp; For (INT I = 0; I <_ numfeature; I ++) {// each feature generates an evenly distributed random number, represented by several rect; numrect = cvfloor (RNG. uniform (double) featureminnumrect, (double) featuremaxnumrect); For (Int J = 0; j <numrect; j ++) {// In rcet (x, y, w, h) random recttemp. In fact, only W and H of _ objectbox are used. In principle, they are evenly distributed in rect (, W, h). recttemp. X = cvfloor (RNG. uniform (0.0, (double) (_ objectbox. width-3); recttemp. y = cvfloor (RNG. uniform (0.0, (double) (_ objectbox. height-3); recttemp. width = cvceil (RNG. uniform (0.0, (double) (_ objectbox. width-recttemp. x-2); recttemp. height = cvceil (RNG. uniform (0.0, (double) (_ objectbox. height-recttemp. y-2); features [I]. push_back (recttemp); weighttemp = (float) Pow (-1.0, cvfloor (RNG. uniform (0.0, 2.0)/SQRT (float (numrect); featuresweight [I]. push_back (weighttemp) ;}} void compressivetracker: samplerect (MAT & _ image, rect & _ objectbox, float _ rinner, float _ router, int _ maxsamplenum, vector <rect> & _ samplebox)/* Description: Compute the coordinate of positive and negative sample image templates arguments:-_ image: processing frame-_ objectbox: recent object position-_ rinner: inner sampling radius-_ router: outer sampling radius-_ maxsamplenum: maximal number of Sampled images-_ samplebox: storing the rectangle coordinates of the sampled images. */{int rowsz = _ image. rows-_ objectbox. height-1; int colsz = _ image. cols-_ objectbox. width-1; float inradsq = _ rinner * _ rinner; // 4*4 float outradsq = _ router * _ router; // 0*0 int Dist; int minrow = max (0, (INT) _ objectbox. y-(INT) _ rinner); // minimum coordinate of the starting position; int maxrow = min (INT) rowsz-1, (INT) _ objectbox. Y + (INT) _ rinner); // coordinates of the maximum starting position; int mincol = max (0, (INT) _ objectbox. x-(INT) _ rinner); int maxcol = min (INT) colsz-1, (INT) _ objectbox. X + (INT) _ rinner); int I = 0; float prob = (float) (_ maxsamplenum)/(maxrow-minrow + 1) /(maxcol-mincol + 1); int R; int C; _ samplebox. clear (); // important rect Rec (0, 0, 0); For (r = minrow; r <= (INT) maxrow; r ++) for (C = mincol; c <= (INT) maxcol; C ++) {// ensure that dists of samplerect (x, y) to _ objectbox (x, y) meet the following requirements: _ router * _ router <Dist <_ rinner * _ rinnerdist = (_ objectbox. y-R) * (_ objectbox. y-R) + (_ objectbox. x-C) * (_ objectbox. x-C); If (RNG. uniform (0 ., 1 .) <prob & Dist <inradsq & Dist> = outradsq) {rec. X = C; Rec. y = r; Rec. width = _ objectbox. width; Rec. height = _ objectbox. height; _ samplebox. push_back (REC); I ++;} _ samplebox. resize (I);} void compressivetracker: samplerect (MAT & _ image, rect & _ objectbox, float _ SRW, vector <rect> & _ samplebox)/* description: compute the coordinate of samples when detecting the object. */{int rowsz = _ image. rows-_ objectbox. height-1; int colsz = _ image. cols-_ objectbox. width-1; float inradsq = _ SRW * _ SRW; int Dist; int minrow = max (0, (INT) _ objectbox. y-(INT) _ SRW); int maxrow = min (INT) rowsz-1, (INT) _ objectbox. Y + (INT) _ SRW); int mincol = max (0, (INT) _ objectbox. x-(INT) _ SRW); int maxcol = min (INT) colsz-1, (INT) _ objectbox. X + (INT) _ SRW); int I = 0; int R; int C; rect Rec (0, 0, 0); _ samplebox. clear (); // importantfor (r = minrow; r <= (INT) maxrow; r ++) for (C = mincol; C <= (INT) maxcol; c ++) {Dist = (_ objectbox. y-R) * (_ objectbox. y-R) + (_ objectbox. x-C) * (_ objectbox. x-C); If (Dist <inradsq) {rec. X = C; Rec. y = r; Rec. width = _ objectbox. width; Rec. height = _ objectbox. height; _ samplebox. push_back (REC); I ++;} _ samplebox. resize (I);} // compute the features of samplesvoid compressivetracker: getfeaturevalue (MAT & _ imageintegral, vector <rect> & _ samplebox, mat & _ samplefeaturevalue) {int sampleboxsize = _ samplebox. size (); _ samplefeaturevalue. create (featurenum, sampleboxsize, cv_32f); float tempvalue; int xmin; int xmax; int ymin; int Ymax; For (INT I = 0; I <featurenum; I ++) {for (Int J = 0; j <sampleboxsize; j ++) {tempvalue = 0.0f; For (size_t K = 0; k <features [I]. size (); k ++) {xmin = _ samplebox [J]. X + features [I] [K]. x; xmax = _ samplebox [J]. X + features [I] [K]. X + features [I] [K]. width; ymin = _ samplebox [J]. Y + features [I] [K]. y; Ymax = _ samplebox [J]. Y + features [I] [K]. Y + features [I] [K]. height; tempvalue + = featuresweight [I] [k] * (_ imageintegral. at <float> (ymin, xmin) + // calculates the sum value of the integral image in the specified region; _ imageintegral. at <float> (Ymax, xmax)-_ imageintegral. at <float> (ymin, xmax)-_ imageintegral. at <float> (Ymax, xmin);} _ samplefeaturevalue. at <float> (I, j) = tempvalue; // calculate the sum value of the integral image in the specified region as a feature ;}}} // update the mean and variance of the Gaussian classifiervoid compressivetracker: classifierupdate (MAT & _ samplefeaturevalue, vector <float> & _ Mu, vector <float> & _ Sigma, float _ learnrate) {scalar mutemp; scalar sigmatemp; For (INT I = 0; I <featurenum; I ++) {meanstddev (_ samplefeaturevalue. row (I), mutemp, sigmatemp); _ Sigma [I] = (float) SQRT (_ learnrate * _ Sigma [I] * _ Sigma [I] + (1.0f-_ learnrate) * sigmatemp. val [0] * sigmatemp. val [0] + _ learnrate * (1.0f-_ learnrate) * (_ Mu [I]-mutemp. val [0]) * (_ Mu [I]-mutemp. val [0]); // equation 6 in paper_mu [I] = _ Mu [I] * _ learnrate + (1.0f-_ learnrate) * mutemp. val [0]; // equation 6 in paper} // compute the ratio classifier void compressivetracker: radioclassifier (vector <float> & _ mupos, vector <float> & _ sigmapos, vector <float> & _ muneg, vector <float> & _ sigmaneg, mat & _ samplefeaturevalue, float & _ radiomax, Int & _ radiomaxindex) {float sumradio; _ radiomax =-flt_max; _ radiomaxindex = 0; float PPOs; float PNEG; int sampleboxnum = _ samplefeaturevalue. cols; For (Int J = 0; j <sampleboxnum; j ++) {sumradio = 0.0f; For (INT I = 0; I <featurenum; I ++) {PPOs = exp (_ samplefeaturevalue. at <float> (I, j)-_ mupos [I]) * (_ samplefeaturevalue. at <float> (I, j)-_ mupos [I])/-(2.0f * _ sigmapos [I] * _ sigmapos [I] + 1e-30 )) /(_ sigmapos [I] + 1e-30); PNEG = exp (_ samplefeaturevalue. at <float> (I, j)-_ muneg [I]) * (_ samplefeaturevalue. at <float> (I, j)-_ muneg [I])/-(2.0f * _ sigmaneg [I] * _ sigmaneg [I] + 1e-30 )) /(_ sigmaneg [I] + 1e-30); sumradio + = Log (PPOs + 1e-30)-log (PNEG + 1e-30); // equation 4} If (_ radiomax <sumradio) {_ radiomax = sumradio; _ radiomaxindex = J ;}} void compressivetracker: Init (MAT & _ frame, rect & _ objectbox) {// compute feature templatehaarfeature (_ objectbox, featurenum); // compute sample templatessamplerect (_ frame, _ objectbox, routerpositive, 0, 1000000, samplepositivebox); samplerect (_ frame, _ objectbox, rsearchwindow * 1.5, routerpositive + 4.0, 100, priority); integral (_ frame, imageintegral, cv_32f); // calculate the integral image; getfeaturevalue (imageintegral, samplepositivebox, priority); getfeaturevalue (imageintegral, samplenegativebox, priority ); classifierupdate (optional, mupositive, sigmapositive, learnrate); classifierupdate (optional, munegative, sigmanegative, learnrate);} void metadata: processframe (MAT & _ frame, rect & _ objectbox) {// encode (_ frame, _ objectbox, rsearchwindow, detectbox); integral (_ frame, imageintegral, cv_32f); getfeaturevalue (imageintegral, detectbox, detectfeaturevalue); int radiomaxindex; float radiomax; random (mupositive, random, munegative, sigmanegative, random, radiomax, radiomaxindex); _ objectbox = detectbox [radiomaxindex]; // random (_ frame, _ objectbox, routerpositive, 0.0, 1000000, samplepositivebox); samplerect (_ frame, _ objectbox, rsearchwindow * 1.5, routerpositive + 4.0, 100, samplenegativebox); getfeaturevalue (imageintegral, samplepositivebox, callback ); getfeaturevalue (imageintegral, samplenegativebox, aggregate); classifierupdate (aggregate, mupositive, sigmapositive, learnrate); classifierupdate (aggregate, munegative, sigmanegative, learnrate );}

References:

Enhanced vision: http://www.cvchina.info/2012/07/31/real-time-compressive-tracking/