Image processing of the foreground detection (c) Code Book (Codebook) (mainly for code upgrade)

Image processing foreground detection (c) Code Book (codebook)
The basic idea of codebook algorithm is to get the time series model of each pixel. This model handles time fluctuations very well, and the disadvantage is that it consumes a lot of memory. The codebook algorithm establishes a codebook (CB) structure for each pixel of the current image, and each codebook structure is composed of multiple codeword (CW).

Specific principles of reference: Vines OPENCV Study Notes (iv) Principle of--codebook algorithm Bicelove codebook algorithm (background modeling)
Code implementation and explanation based on C interface: zcube "Learning OpenCV" routine parsing ex_9_3 (Codebook model to achieve background subtraction)
Here, the corresponding portions of the above code are upgraded to become C + + interfaces. The environment is vs2015+opencv3.2 contrib version.

/************************************************************************//* A few more thoughts on Codebook MoD Els in general, the Codebook method works quite well across a wide number of conditions, and it's relatively quick to tra In and to run. It doesn ' t deal well with varying patterns of light-such as morning, noon, and evening sunshine-or with someone turnin G lights on or off indoors. This type of global variability can is taken into account by using the several different codebook models, one for each conditi                                       On, and then allowing the condition to control which model is active. *//************************************************************************/#include <iostream> #include
<opencv.hpp> using namespace std;

using namespace CV; #define CHANNELS 3//Set the number of image channels processed, requiring a channel number less than or equal to the image itself///////////////////////////////////////////////////////////////////  The following is the code of the code element of the data structure//processing the image of each pixel corresponding to a codebook, each code book can have a number of code elements
When it comes to a new field, you will usually encounter some strange nouns, do not be frightened by these nouns, in fact the idea is simple typedef struct CE {uchar learnhigh[channels];     High side threshold for learning//This code element limits the threshold of each channel (learning bounds) Uchar Learnlow[channels]; Low side threshold for learning//The minimum threshold for each channel of this code////The value of each channel in the learning process if a new pixel x[i], there is learnlow[i]<=x[i]&lt          ; =learnhigh[i], then the pixel can be merged in this code element Uchar Max[channels];          High side of box boundary//The maximum value of each channel in pixels belonging to this code Uchar Min[channels];          Low side of box boundary//The minimum value of each channel in pixels belonging to this code int t_last_update; This is the book keeping to allow us to kill stale entries//The last time this code was updated, each frame is a unit time, used to calculate stale int stale                  ;                     Max Negative run (biggest period of inactivity)//This code element maximum does not update time, is used to delete the specified time does not update the code element, the compact code this} code_element;
	The data structure of the code element typedef struct CODE_BOOK {code_element **cb;
	The two-dimensional pointer of the code element, which is understood as a pointer to the array of code element pointers, makes it unnecessary to copy the code element when adding the code element, just to assign the value of the simple pointer to int numentries; In this Code bookThe number of code elements int T;                         Count every access//This code is now in time, one frame for a time unit} CodeBook; Data structure of codebook///////////////////////////////////////////////////////////////////////////////////// int Updatecodebook (Uchar *p, CodeBook &c, unsigned cbbounds)//Updates the CodeBook entry with a new data  
									Point////P Pointer to a YUV pixel//C Codebook for this pixel Cbbounds Learning bounds for codebook (Rule of Thumb:10)//numchannels number of color Chan  
									Nels we ' re learning////NOTES://Cvbounds must be of size cvbounds[numchannels] RETURN//codebook index int updatecodebook (Uchar *p, codebook &c, unsigned *
	cbbounds, int numchannels) {if (c.numentries = = 0) c.t = 0;   Code in this code element is zero initialization time is 0 c.t + = 1; Record Learning Event//Each call is added one at a time, that is, each frame image plusA//set high and a low BOUNDS int n;
	unsigned int high[3], low[3];
		for (n = 0; n<numchannels; n++) {High[n] = * (p + N) + * (Cbbounds + N);
		* (P+n) and p[n] results equivalent, tested * (P+N) faster if (High[n] > 255) high[n] = 255;
		Low[n] = * (p + N)-* (Cbbounds + N);
		if (Low[n] < 0) Low[n] = 0;
	Using the pixel channel data of p, plus and minus the value in Cbbonds, as the upper and lower bounds of this pixel threshold}//see IF this FITS an EXISTING codeword int matchchannel;
	int i;
		for (i = 0; i<c.numentries; i++) {//Traverse this code for each code element, test whether p pixels satisfy one of them matchchannel = 0; for (n = 0; n<numchannels; n++)//traversal per channel {if ((C.cb[i]->learnlow[n] <= * (p + N)) && (* (p + N)  
			<= C.cb[i]->learnhigh[n])//found A entry for the This channel//if the P-pixel channel data is between the lower and upper bounds of the code element threshold
			{matchchannel++; }} if (Matchchannel = = numchannels)//If an entry is found over all channels//if p pixels each channel satisfies the above
			condition {c.cb[i]->t_last_update = c.t; Update the code element time for the current time//adjust this CodewOrd for the first channel for (n = 0; n<numchannels; n++)//Adjust the code element each channel maximum minimum value {if (C.cb[i]->max[n] &l T
				* (p + N)) c.cb[i]->max[n] = * (p + N);
			else if (C.cb[i]->min[n] > * (p + N)) c.cb[i]->min[n] = * (p + N);
		} break;  
							}}//ENTER A new code word if NEEDED if (i = = c.numentries)//No existing CODE word found, make A new one
		P pixels do not satisfy any one of this code, create a new code element {Code_element **foo = new Code_element*[c.numentries + 1]; Apply c.numentries+1 A pointer to the code element for (int II = 0; ii<c.numentries; ii++)//To point the previous c.numentries pointer to each code element that already exists foo[ii

		] = C.cb[ii];
		Foo[c.numentries] = new Code_element;
		Apply for a new code element if (c.numentries) delete[] C.CB;
		Delete C.CB pointer array c.cb = FOO; Assign the Foo head pointer to C.CB for (n = 0; n<numchannels; n++)//Update new code element each channel data {c.cb[c.numentries]->learnhigh[n] = h
			Igh[n];
			C.cb[c.numentries]->learnlow[n] = Low[n];
			C.cb[c.numentries]->max[n] = * (p + N); C.cb[c.numentries]->min[n] = * (p + N);
		} c.cb[c.numentries]->t_last_update = c.t;
		C.cb[c.numentries]->stale = 0;
	C.numentries + = 1; }//OVERHEAD to track potential STALE ENTRIES for (int s = 0; s<c.numentries; s++) {//This garbage are to Tra
		CK which codebook entries is going stale int negrun = c.t-c.cb[s]->t_last_update;
	Calculates the non-update time of the code element if (C.cb[s]->stale < Negrun) C.cb[s]->stale = Negrun;  
	}//Slowly adjust learning BOUNDS for (n = 0; n<numchannels; n++)//If the pixel channel data is within the high and low threshold range, but outside the code element threshold, slowly adjust this code element learning limit
		{if (C.cb[i]->learnhigh[n] < high[n]) C.cb[i]->learnhigh[n] + = 1;
	if (C.cb[i]->learnlow[n] > Low[n]) c.cb[i]->learnlow[n]-= 1;
} return (i); }/////////////////////////////////////////////////////////////////////////////////////Uchar CvbackgroundDiff ( Uchar *p, CodeBook &c, int minmod, int maxmod)//Given a pixel and a code book, determine if the pixel is covered B       Y the Codebook////P Pixel pointer (YUV interleaved)//C Codebook Reference//numchannels number of channels we are testing//  Maxmod Add This (possibly negative) number onto Max level when code_element determining if new pixel is foreground//   
Minmod Subract This (possible negative) number from min level code_element when determining if pixel is foreground//  
NOTES://Minmod and Maxmod must have length numchannels, e.g. 3 channels = Minmod[3], maxmod[3]. Return//0 = background, 255 = foreground Uchar Backgrounddiff (Uchar *p, CodeBook &c, int Numchan
	Nels, int *minmod, int *maxmod) {//The following steps and background learning find the code element in the same int matchchannel;
	See IF the FITS an EXISTING codeword int i;
		for (i = 0; i<c.numentries; i++) {matchchannel = 0; for (int n = 0; n<numchannels; n++) {if ((C.cb[i]->min[n]-minmod[n] <= * (p + N)) && (* (p + N) &lt ; = C.cb[i]->max[n] + maxmod[n])) matchchannel++; Found an entry for this ChannEl else break; } if (Matchchannel = = numchannels) break;

	Found an entry this matched all channels} if (i = = c.numentries)//p pixels each channel value satisfies one of the codes in the Codebook, then returns the white return (255);
return (0); }//utilites//////////////////////////////////////////////////////////////////////////////////////////////////// int clearstaleentries (CodeBook &c)// After your ' ve learned for some period of time, periodically call this to clear out stale codebook entries////c Co Debook to clean up////Return//number of entries cleared int clearstaleentries (CodeBook &c) {int Staleth           Resh = c.t >> 1; Set refresh time int *keep = new Int[c.numentries];                    Request an array of tokens int keepcnt = 0; Number of records not deleted//see which CODEBOOK ENTRIES is TOO STALE for (int i = 0; i<c.numentries; i++)//Traverse code Book for each code element {if (C.cb[i]->stale > Stalethresh)//If not updated in the code elementTime is greater than the set refresh time, then marked as delete keep[i] = 0;
		Mark for destruction else {keep[i] = 1,//mark to keep keepcnt + = 1;                        }}//KEEP only the good c.t = 0;
	Full reset on stale tracking//code ben time 0 Code_element **foo = new CODE_ELEMENT*[KEEPCNT];
	Request an array of code element pointers of size keepcnt int k = 0;
			for (int II = 0; ii<c.numentries; ii++) {if (Keep[ii]) {foo[k] = C.cb[ii];       Foo[k]->stale = 0;
			We have to refresh these entries for next clearstale foo[k]->t_last_update = 0;
		k++;
	}}//clean up delete[] keep;
	Delete[] C.CB;
	C.CB = foo;
	Assign the Foo head pointer address to C.CB int numcleared = c.numentries-keepcnt;
	Number of code elements cleaned c.numentries = keepcnt;
The remaining code element address return (numcleared);
	} int main () {//////////////////////////////////////////Required variable Videocapture capture ("e://footage//2.mp4");
	Mat Rawimage;
	Mat Yuvimage;
	Mat Imaskcodebook;
	Codebook* CB;
	unsigned cbbounds[channels]; uchar* PcoloR
	YUV pointer int imagelen;
	int nchannels = CHANNELS;
	int Minmod[channels];

	int Maxmod[channels];
	Initialize each variable Namedwindow ("Raw");



	Namedwindow ("CodeBook");
	Capture >> Rawimage;
	Mat yuvimage (Rawimage.rows, Rawimage.cols, 8, 3);
	Assign a yuvimage to the same size as the Rawimage, 8-bit 3-channel image Mat Imaskcodebook (rawimage.rows, Rawimage.cols, CV_8UC1, Scalar (255)); 
	Assigning a imaskcodebook to the same size as the Rawimage, the 8-bit single-channel image//sets all elements of the single-channel array to 255, which is initialized to a white image Imagelen = Rawimage.cols * rawimage.rows;
	CB = new Codebook[imagelen];
	Get the same set of codebook as the number of pixels in the image so that each pixel is processed for (int i = 0; i<imagelen; i++)//Initialize 0 cb[i].numentries per code element = 0;   for (int i = 0; i<nchannels; i++) {cbbounds[i] = 10;   Minmod[i] = 20 for determining the threshold of each channel of the code element;   For the background difference function maxmod[i] = 20; Adjust its value to achieve the best segmentation}////////////////////////////////////////////////////////////////////////////Start processing video each frame diagramLike for (int i = 0;; i++) {capture >> rawimage;
		Cvtcolor (Rawimage, Yuvimage, CV_BGR2YCRCB);  

		Color space conversion, the conversion of rawimage to YUV color space, output to yuvimage//Even if the effect is still very good//Yuvimage = Cvcloneimage (rawimage);
			if (I <= 30)//30 frames for background learning {Pcolor = (Uchar *) (yuvimage.ptr (0)); Channel data for yuvimage image for (int c = 0; c<imagelen; C + +) {Updatecodebook (Pcolor, Cb[c], cbbounds, nchannels)
				;
				For each pixel, call this function to capture the relevant change image in the background pcolor + = 3; 3-channel image, pointing to the next pixel channel data} if (i = = 30)//to 30 frames call the following function, delete the stale code element {for (int c = 0; c<imagelen; C + +
			) clearstaleentries (Cb[c]);
			}} else {Uchar Maskpixelcodebook; Pcolor = (Uchar *) (yuvimage.ptr<uchar> (0)); 3 Channel YUV image Uchar *pmask = (Uchar *) (imaskcodebook.ptr (0)); 1 channel image//points to the first element of the Imaskcodebook channel data series for (int c = 0; c<imagelen; C + +) {Mask Pixelcodebook = Backgrounddiff (Pcolor, Cb[c], nchannels, Minmod, Maxmod);
				I saw this childhood enlightened, began to understand codebook hehe *pmask++ = Maskpixelcodebook;
				Pcolor + = 3;
		Pcolor points to a 3-channel image}} imshow ("Raw", rawimage);


		Imshow ("CodeBook", Imaskcodebook);
		Waitkey (20);
		if (i ==450) waitkey (0);
	cout << "This is the processing of the section" << I << "frame image data \n\r";

	} delete[] CB;
return 0; }

Final Test results:
SOURCE Program C Interface:


Upgrade program C + + interface

The results are basically the same, no difference.