Mosaic processing of partial video regions

In TV interviews, sometimes some interviewees are reluctant to show up. In this case, the interviewee may face the camera. But more often, the interviewee still faces the camera and mosaic the face of the interviewee while broadcasting the TV program. This mosaic process makes it impossible for the audience to see the real face of the interviewee, so as to meet the original intention of the interviewee not to show up. As a programmer, have you ever thought about how to handle this effect? This article introduces a simple and easy programming method for video mosaic processing in the local area.

I.Mosaic Processing Principle and Implementation

First, let's take a look at the comparison effect of the same video image before and after Mosaic processing. 1.

Figure 1 Comparison of effects of face Mosaic

After mosaic processing, you cannot identify her real face, right? So how can the mosaic effect appear? As you know, images are composed of pixels. The size of pixels determines the precision of the image (that is why small-sized TVs look clearer than large-sized TVs ). If we zoom in the pixels in the specified area, will the mosaic effect appear? Don't worry, here is a key question: for a given display device, its pixel particle size is physically unchangeable. How can we zoom in ?!
There is a way: when several adjacent pixels are used to present the same pixel value at the same time, isn't it equal to enlarging the pixel? If we zoom in each pixel in a specified area, the Mosaic Area will be larger than the original one (if the width of the specified area is W, if the ratio of horizontal pixel enlargement is ratiox, the area width after Mosaic processing will overwrite w x ratiox ). How can we limit the mosaic processing region to the region specified by the user? The author's practice is to perform a subsample of pixels in a specified area. 2. Suppose we will perform mosaic processing on the R1 area of the image.

Figure 2 mosaic Processing Area

Assume that the pixels in the R1 region are 3:

Figure 3 pixels in the R1 Region

Assuming that the horizontal amplification ratio (ratiox) of the pixel is 3 and the vertical amplification ratio (ratioy) is also 3, the pixel value distribution at each corresponding position is 4 after Mosaic processing:

Figure 4 pixels in the R1 region after Mosaic Processing

We can see that in the R1 region, sampling is performed once per three pixels in the horizontal direction (p00, P03, p06, p09, P30, p33, p36, p39, P60, p63, P66, and p69 are all sampling points ), in the vertical direction, each sample pixel row repeats three times (2nd and 3 rows copy 1st rows of content, 5th and 6 rows copy 4th rows of content, and so on ); each sample point pixel is enlarged to a 3x3 macro block, that is, the sample point pixel is enlarged 9 times.

C ++ Implementation of mosaic processing in the specified Image Area

// Image frame Data Pointer
Pbyte pimage;
// Obtain image data
//...
// Pointer to the beginning of the first line of the image
Pbyte pimagetopline = NULL;
// Image span (in bytes)
Long imagestride = 0;
// If the image data is stored in the scanning order from bottom to top,
// The first row of the image should be the last 1st rows of pimage data;
// If the image data is stored in the order of scanning from top to bottom,
// The second row of the image is the position indicated by pimage.
If (m_bisbottomup)
{
Imagestride =-m_nimagestride;
Pimagetopline = pimage + m_nimagestride * (m_nimageheight-1 );
}
Else
{
Imagestride = m_nimagestride;
Pimagetopline = pimage;
}

// Ratiox is the magnification of the Water square up the pixel
// Ratioy is the magnification of pixels in the vertical direction
// Maskstride indicates the width of the mosaic processing area (in bytes)
/* Macrowidth and macorheight are calculated as follows:
Rect m_maskrect; // The area of the rectangle to be Mosaic (specified by the user)
Int maskwidth = m_maskrect.right-m_maskrect.left + 1;
Int maskheight = m_maskrect.bottom-m_maskrect.top + 1;
Macrowidth = maskwidth/m_nratiox;
Macroheight = maskheight/m_nratioy;
*/
Int macrowidth, macroheight, maskstride, ratiox, ratioy;
// Mosaic processing:
// Pmaskpixel points to the current pixel,
// Pmaskline points to the current row,
// The next line of pmasknextline
Pbyte pmasktopline, pmaskline, pmasknextline, pmaskpixel;
// Pmasktopline points to the 1st rows of the region for Mosaic Processing
// Note: m_npixelbytes indicates the number of bytes occupied by a single pixel.
Pmasktopline = pimagetopline + m_maskrect.top * imagestride + m_maskrect.left * m_npixelbytes;
Macrowidth = m_nmacrowidth;
Macroheight = m_nmacroheight;
Maskstride = m_nmaskstride;
Ratiox = m_nratiox;
Ratioy = m_nratioy;

// Scan pixels in the specified area for Mosaic processing...
Int cycle = 0;
For (INT I = 0; I <macroheight; I ++)
{
// Locate the current row for Mosaic Processing
Pmaskline = pmasktopline + I * ratioy * imagestride;
// Locate the current pixel that requires mosaic Processing
Pmaskpixel = pmaskline;
For (Int J = 0; j <macrowidth; j ++)
{
// Pixel amplification in the horizontal direction
For (cycle = 0; cycle <ratiox-1; cycle ++)
{
// Copy the current pixel value to the next pixel on the right
Memcpy (pmaskpixel + m_npixelbytes, pmaskpixel, m_npixelbytes );
// Point to the next Pixel
Pmaskpixel + = m_npixelbytes;
}
// Point to the next sample Pixel
Pmaskpixel + = m_npixelbytes;
}

// Pixel amplification in the vertical direction
For (cycle = 0; cycle <ratioy-1; cycle ++)
{
// Obtain the next row pointer of the mosaic Processing Area
Pmasknextline = pmaskline + imagestride;
// Copy the current row of the mosaic processing area (mosaic processing completed) to the next row.
Memcpy (pmasknextline, pmaskline, maskstride );
// Modify the current row pointer to the next row
Pmaskline = pmasknextline;
}
}

Ii. component development and demonstration

With the implementation of mosaic processing algorithms, the next question is, how can we obtain continuous video image frame data? Here, we can use graphedit (bin/dxutils/graphedt.exe In the SDK directory), a tool that comes with the DirectX SDK ). Run graphedit, 5:

Figure 5 graphedit tool software

Execute the menu command file | render media file ..., In the pop-up dialog box that appears later, select a multimedia file (for example, select an MPEG2 file mp2_sales.mpg with a fixed portrait and position) and automatically build 6 links:

Figure 6 playback link built using graphedit

Then run the menu command graph | play to play the mp2_sales.mpg file. Execute graph | pause or graph | stop to pause or stop the current playback.
Note that graphedit uses the DirectShow technology to play the mp2_sales.mpg file! As you know, DirectX is a set of programming interfaces provided by Microsoft to develop high-performance graphics, sound, input, output, and online games on Windows platforms. DirectShow is a member of DirectX, it is used for audio and video data collection, multimedia file playback, and other applications. The most basic functional module in DirectShow is filter (each rectangle block in Figure 6 represents a filter). Each filter has at least one pin for receiving or outputting data; filter always provides certain functions (in figure 6, the first filter on the left is the file source, and the MPEG-2 splitter is responsible for separating the audio and video in the MPEG2 data stream, cyberlink audio decoder is responsible for decoding audio data in MPEG format, HQ MPEG-2 video decoder is responsible for decoding video data in MPEG format, default directsound device is responsible for audio playback, video Renderer is responsible for video display ); various filters are connected in sequence to collaborate with each other. Data flows along the Arrow between filters until default directsound device and video Renderer.
DirectShow is a modular and open application framework. We can develop our own filter component and insert it to a location in the filter link to obtain the opportunity to process data streams. Taking the video mosaic processing that needs to be implemented in this article, we can implement the mosaic Processing Algorithm in a filter and connect it to the video decoding filter, to obtain continuous, non-compressed image frame data. We name this filter "HQ video Mosaic". Because this filter can modify data "locally" on the input image frame, the Trans-in-place model can be used for the filter; the filter accepts 16-bit, 24-bit, and 32-bit RGB data input. After HQ video mosaic is developed, the hqmosaic. Ax file is generated (assuming it is stored in C:/, and then registered using regsvr32.exe of the system (the method is to execute the command line regsvr32 C:/hqmosaic. Ax ). (Note: For more details about the DirectShow filter development method, I will not start it here; if you are interested, you can refer to the author's two collections for "DirectShow Development Guide" and "DirectShow practical selection". HQ video mosaic filter source code, please download to the http://hqtech.nease.net .)
After the filter component is developed and successfully registered, it can be used in graphedit. The first step is to build the 6 filter link. Then execute the menu command graph | insert filters ..., In the displayed dialog box, open the "DirectShow filters" directory and double-click "HQ video Mosaic" to add it. Next, disconnect the HQ MPEG-2 video decoder from the video Renderer (press the Arrow between the two filters and press the delete key on the keyboard ). Connect HQ MPEG-2 video decoder to HQ video mosaic, and then connect HQ video mosaic to video Renderer. (Connection method between two filters: first, press and hold the left mouse button on the output pin of the filter to be connected, and drag the mouse to the input pin of the next filter, finally, open the left mouse button .) Final filter link 7:

Figure 7 use the gradient filter in graphedit

Now, execute the menu command graph | play, and we can see a demo of mosaic processing in the local area of the video similar to figure 1. In addition, go to the HQ video mosaic filter attribute page (to open the attribute page: Select HQ video mosaic and right-click the menu item "Filter Properties ..."), We can also dynamically update the mosaic processing areas and the ratio of horizontal/vertical pixel amplification.

Figure 8 property page of HQ video mosaic Filter

Iii. Summary

This article introduces the principle of video mosaic processing and a C ++ algorithm. Then, with the help of DirectShow, this article also completed the demonstration of the effect of video Partial Area mosaic processing.