"Computer vision" extracts the foreground object from the video

Foreground extraction and target detection

When observing the scene is a fixed camera, the background is almost unchanged. In this case, the element of interest is the object moving in the scene. In order to extract these foreground objects, we need to model the background and then compare the models of the current frame with the background model to detect the foreground object. foreground extraction is a fundamental step in intelligent monitoring applications.
OPENCV's video module contains several more commonly used background subtraction methods, in which the mixed Gaussian model (Gaussian of Mixture Models, GMM) has a good effect.

The common target detection methods are inter-frame difference and background subtraction.
The key of the background subtraction method is to establish a robust background model (background image), commonly used to set up background model methods: mean value method, median method, sliding average filter method, Gaussian; mixed Gaussian model, codebook, etc.

Background minus method

The meaning of the background subtraction method is to construct a reference image (a pure background), subtract each new video frame from the reference picture, and make a binary value to get the method of moving foreground. This means that the result of the background subtraction method is a highlighted representation of the non-static region. The simplest way to build a reference image is to average a series of background images on a continuous period of time. There are many problems with this approach, and it also requires a sequence of pictures with no foreground and only a pure background as a training sample. After the training, the movement of the background and the stationary movement of the foreground of the training stage are regarded as the moving targets. In addition, the method cannot cope with the gradual change of light in the scene.

The principle of mixed Gaussian model

First, the hybrid Gaussian algorithm maintains multiple models (that is, multiple sliding averages) for each pixel. Therefore, if the background pixel changes back and forth between two values, then two sliding averages are stored. If the new pixel value does not belong to one of them, it is considered to be a foreground value.
Second, not only the sliding average, but also the sliding variance is saved. The calculated mean and variance form a Gaussian model through which we can learn the probability that a pixel value belongs to the model. It makes it easier to determine the appropriate threshold because it now behaves as a probability rather than an absolute value of the difference. at the same time, the area with obvious change in the background, the difference of the foreground object will be much larger.
Finally, when the Gaussian model is hit frequently enough, it is removed from the current model. Conversely, if a pixel value is outside the currently maintained background model (that is, it is a foreground pixel), then the new Gaussian model will be created. If it is hit frequently later, it becomes part of the background.

Theory

Mixed Gaussian background modeling is a background representation based on the statistical information of the pixel sample, which uses the statistical information such as the probability density of a large number of sample values over a long period of time (such as the number of patterns, the mean and standard deviation of each pattern), and then uses the statistical difference (such as the 3σ principle) to determine the target pixel. The complex dynamic background can be modeled and the computational amount is large.
In the mixed Gaussian background model, it is considered that the color information between pixels is irrelevant, and the processing of each pixel is independent of each other. For each pixel in the video image, the change of its value in the sequence image can be regarded as the random process of continuously producing the pixel value, that is, the Gaussian distribution is used to describe the color rendering law of each pixel point.
For the multi-peak Gaussian distribution model, each pixel of the image is modeled by superposition of multiple Gaussian distributions of different weights, and each Gaussian distribution corresponds to a state in which the color of the pixel can be produced, and the weights and distribution parameters of each Gaussian distribution are updated over time. When processing color images, it is assumed that the image pixel points R, G, b tri-color channels are independent of each other and have the same variance.

Interface of Gaussian mixture model in OPENCV

Two versions of the Gaussian mixed background/foreground segmentation method are implemented in OPENCV.

Backgroundsubtractormog

Constructors can use either a default constructor or a constructor with a formal parameter:

BackgroundSubtractorMOG::BackgroundSubtractorMOG()BackgroundSubtractorMOG::BackgroundSubtractorMOG(intintdoubledouble noiseSigma=0)

Where history is the number of historical frames used, Nmixtures is the mixed Gaussian number, Backgroundratio is the background scale, and noisesigma is the noise weight.

The interface that is called is overloaded with only the operator ():

void BackgroundSubtractorMOG::operatordouble learningRate=0)

Where image is the current frame, Fgmask is the output foreground mask,learningrate is the background learning rate.

BackgroundSubtractorMOG2

constructors can use either a default constructor or a constructor with a formal parameter :

BackgroundSubtractorMOG2::BackgroundSubtractorMOG2()  BackgroundSubtractorMOG2::BackgroundSubtractorMOG2(int history,floatbool bShadowDetection=true

History Ibid. To use the number of historical frames; The varthreshold represents the threshold value used on the Markov squared distance (this value does not affect the background update rate); Bshadowdetection Indicates whether shadow detection is used (if shadow detection is turned on, Use 127 for shading in mask).

Use the overloaded operator () to invoke each frame detection function :

void BackgroundSubtractorMOG2::operatordouble learningRate=-1)

The parameter meaning is the same as the operator () function in Backgroundsubtractormog.

BackgroundSubtractorMOG2 also provides the Getbackgroundimage () function to return the background image :

void BackgroundSubtractorMOG2::getBackgroundImage(OutputArray backgroundImage)

Sample code

intMain () {Videocapture Video (0);       Mat frame,mask,thresholdimage, output; BackgroundSubtractorMOG2 Bgsubtractor ( -, -,true); Namedwindow ("Background"); while(true) {video>>frame; Bgsubtractor (Frame,mask,0.001); Imshow ("Background", mask);Charc = (Char) Waitkey ( -);if(c = = -) Break; }return 0; }

Resources

Mixed Gaussian background model and OPENCV implementation
The principle of mixed Gaussian algorithm in OpenCV

reprint Please indicate the author Jason Ding and its provenance
Gitcafe Blog Home page (http://jasonding1354.gitcafe.io/)
GitHub Blog Home page (http://jasonding1354.github.io/)
CSDN Blog (http://blog.csdn.net/jasonding1354)
Jane Book homepage (http://www.jianshu.com/users/2bd9b48f6ea8/latest_articles)
Baidu Search jasonding1354 access to my blog homepage

"Computer vision" extracts the foreground object from the video