Image processing--motion detection

Today see a motion detection algorithm, see his effect seems to be good, free to study

Download demo-82.5 KB Download source-114 kb

Introduction

There are many approaches for motion detection in continuous the video streams. All of them are based on comparison of current video frame with one from the previous frames or with something that we ' ll Call background. In this article I ' ll try to describe some of the most common approaches.

In describing the algorithms, I'm ' ll use an image processing library I ' ve described in my previous article. So, if you are are common with it, it'll only help.

The demo applications support the following types of video Sources:avi files (using a for Windows, interop library is Included); Updating JPEG from Internet cameras; MJPEG (motion JPEG) streams from different internet cameras; MMS Stream-microsoft Media Services; Local capture device (USB cameras or other capture devices). Algorithms

One of the most common approaches is to compare the current frame with the previous one. It's useful in a compression when you need to estimate changes and to write only the changes, not the whole frame. But It is isn't the best one for motion detection applications. So, let me describe is more closely.

Assume that we have the original BPP RGB image called current frame (image), a grayscale copy of it (Currentframe) and a Previous video frame also gray scaled (backgroundframe). The regions where these two frames are differing a bit. For the purpose we can use difference and Threshold filters.

Create filters
Difference differencefilter = new difference ();
IFilter thresholdfilter = new Threshold (255);
Set Backgroud frame as a overlay for difference filter
differencefilter.overlayimage = backgroundframe;
Apply the filters
Bitmap tmp1 = differencefilter.apply (currentframe);
Bitmap TMP2 = thresholdfilter.apply (TMP1);

On this step we'll get a image with white pixels on the place where the current frame is different from the previous Fram E on the specified threshold value. It ' s already possible to count the pixels, and if the amount of it are greater then a predefined alarm level we can signal About Motion event.

But most cameras produce a noisy image, so we'll get motion in such places, where there are no motion at all. To-remove random noisy pixels, we can use a erosion filter, for example. So, we'll get to now mostly only the regions where there is actual motion.

Create filter
IFilter erosionfilter = new Erosion ();
Apply the filter 
Bitmap Tmp3 = erosionfilter.apply (TMP2);

The simplest motion detector is ready! We can highlight the motion regions if needed.

Extract red channel from the original image
IFilter Extrachchannel = new Extractchannel (RGB. R);
Bitmap Redchannel = extrachchannel.apply (image);
Merge  Red channel with motion regions
Merge mergefilter = new merge ();
Mergefilter.overlayimage = Tmp3;
Bitmap Tmp4 = mergefilter.apply (Redchannel);
Replace red channel in the original image
Replacechannel Replacechannel = new Replacechannel (RGB. R);
Replacechannel.channelimage = Tmp4;
Bitmap TMP5 = replacechannel.apply (image);

Here are the result of it:

From the above we can and disadvantages of the approach. If the object is moving smoothly we'll receive small changes from frame to frame. So, it's impossible to get the whole moving object. Things become worse, when the "object is" moving so slowly, when the algorithms won't be give any.

The

There is another approach. It's possible to compare the "current frame is not" with the previous one but with the "the" in the "the" video sequence. So, if there were no objects in the initial frame, comparison of the ' current ' frame with the ' the ' Le moving object independently of its motion speed. But, the approach have a big disadvantage-what would happen, if there is, for example, a car on the "the", But the n It is gone? Yes, we ll always have motion detected on the place, where the car is. Of course, we can renew the initial frame sometimes, but still it would not be give us good results in the cases where we can Not guarantee, that the "the" the "the" ' contain only static background. But, there can be the inverse situation. If I ' ll put a picture on the wall in the room? I ' ll get motion detected until the initial frame would be renewed.

The most efficient algorithms are based on building so called backgrounds of the scene and comparing all current frame With the background. There are many approaches to build the scene, but most of them are-too. I ' ll describe here's my approach for building the background. It ' s rather simple and can is realized very quickly.

As in the previous case, let's assume that we have a original BPP RGB image called current frame (image), a Graysca Le copy of it (Currentframe) and a background frame also gray scaled (backgroundframe). At the beginning we get the "the" "the" the "the" sequence as the background frame. And then we ll always compare the current frame with the background one. But it would give us the result I ' ve described above which we obviously don ' t want much. Our approach are to "move" the background frame to the current frame on the specified amount (I ' ve used 1 level per frame). We move the background frame slightly at the direction of the current frame-we are changing colors of pixels in the BAC Kground frame by one level per frame.

Create filter
movetowards movetowardsfilter = new Movetowards ();
Move background towards the current frame
movetowardsfilter.overlayimage = currentframe;
Bitmap tmp = movetowardsfilter.apply (backgroundframe);
Dispose old background
backgroundframe.dispose ();
Backgroundframe = tmp;

And now, we can use the same approach we ' ve used above. But, let me extend it slightly.

Create processing filters sequence
filterssequence processingfilter = new Filterssequence ();
Processingfilter.add (New Difference (backgroundframe));
Processingfilter.add (new Threshold (255));
Processingfilter.add (New Opening ());
Processingfilter.add (New Edges ());
Apply the filter
Bitmap TMP1 = processingfilter.apply (currentframe);

Extract red channel from the original image
IFilter Extrachchannel = new Extractchannel (RGB. R);
Bitmap Redchannel = extrachchannel.apply (image);
Merge  red channel with moving object borders
Merge Mergefilter = new merge ();
Mergefilter.overlayimage = TMP1;
Bitmap TMP2 = mergefilter.apply (Redchannel);
Replace red channel in the original image
Replacechannel Replacechannel = new Replacechannel (RGB. R);
Replacechannel.channelimage = TMP2;
Bitmap Tmp3 = replacechannel.apply (image);

Now it's looks much better!

There is another approach based on the idea. As in the previous cases, we have a original frame, and a gray scaled version of it and the background frame. But Let's apply pixellate filter to the "current frame" and to the background before further processing.

Create filter
IFilter pixellatefilter = new Pixellate ();
Apply the filter
Bitmap newimage = pixellatefilter (image);

So, we have pixellated versions to the current and background frames. Now, we need the background frame towards the current frame as we were doing before. The next change are only in the main processing step:

Create processing filters sequence
filterssequence processingfilter = new Filterssequence ();
Processingfilter.add (New Difference (backgroundframe));
Processingfilter.add (new Threshold (255));
Processingfilter.add (New dilatation ());
Processingfilter.add (New Edges ());
Apply the filter
Bitmap TMP1 = processingfilter.apply (currentframe);

After merging TMP1 image and the red channel of the original image, we ' ll get the next:

May is it looks perfect as the previous one, but the approach has the great possibility for performance Optimizatio N. conclusion

I ' ve described only ideas here. To use this ideas in the real applications, the need to optimize its realization. I ' ve used an image processing library for simplicity, it's not a video processing library. Besides, the library allows me to a different areas more quickly, than to write optimized solutions Ing. A small sample of optimization can be found in the sources.

Andrew Kirillov

Click here to view Andrew Kirillov's online profile.