Discussion on developing video conferencing software based on P2P Technology

This article aims to analyze and develop main technologies related to video conferencing software based on P2P technology, and give a simple example.

I. Introduction

I believe most people have heard of Microsoft's NetMeeting and even some people have used it directly. Today, many online worms are addicted to video chat. How is this kind of software developed? In this article, let's analyze and develop the main technologies related to video conferencing software based on P2P technology, and give a concise example. This example application allows two people on the LAN/Intranet to hold video meetings.

Intuitively, we will know that the major problem involved in developing such software is that the large size of video frames will greatly affect the quality of data transmission. Therefore, the performance of such software mainly depends on the quality of video frame encoding and decoding. To this end, in this example, we chose the fast H.263 encoder library, which has a very good compression ratio, which effectively overcomes the contradiction between our speed in image transmission.

Please note that interested readers can slightly modify the sample program in this article to apply it to the Internet environment.

Ii. Audio Recording and playback Problems

This part of development is relatively simple. First, the APIs for this function have been provided since Windows 3.1 (winmm. Lib + mmsystem. h). Second, we can find the ready-made packaging classes through the convenient Internet. In this article, we borrowed two existing recordsound and playsound classes. These two classes are derived from the cwinthread class, and you can use them "easily. The following code demonstrates the use of these two classes. For more information about the definition of the packaging class, see download the source code file.

// Create and start the recording thread

Record = new recordsound (this );

Record-> createthread ();

// Create and start the playing thread

Play = new playsound1 (this );

Play-> createthread ();

// Start recording

Record-> PostThreadMessage (WM_RECORDSOUND_STARTRECORDING, 0, 0 );

// Start playing

Play-> PostThreadMessage (WM_PLAYSOUND_STARTPLAYING, 0, 0 );

// During audio recording, we can use OnSoundData in the RecordSound class

// Use the data in the callback function. Here, you can place the data you want to send to the remote host ......

// Play the audio data received from the remote host

Play-> PostThreadMessage (WM_PLAYSOUND_PLAYBLOCK, size, (LPARAM) data );

// Stop recording

Record-> PostThreadMessage (WM_RECORDSOUND_STOPRECORDING, 0, 0 );

// Stop playing

Play-> PostThreadMessage (WM_PLAYSOUND_STOPPLAYING, 0, 0 );

// Finally, stop the recording thread

Record-> PostThreadMessage (WM_RECORDSOUND_ENDTHREAD, 0, 0 );

// Stop the playing thread

Play-> PostThreadMessage (WM_PLAYSOUND_ENDTHREAD, 0, 0 );

Annotations have been added above, and the usage is clear at a glance.

Iii. Video Capture Problems

Currently, two video applications are generally developed on Windows. One is based on the secondary software development kit SDK attached to the video capture card. Advantages of this method: it is easy to get started by directly applying existing APIs with complete information, but its disadvantages are also obvious: Strong hardware dependence and lack of proper flexibility. Therefore, it cannot fully meet the needs of developing common video applications.

Another solution is based on Microsoft's VFW (Video for Windows. This SDK provides ready-made software kits (a set of Apis) for developing video applications on Windows platforms ), developers can use these functions to conveniently implement video capture, video editing, and video playback. In particular, they can use the built-in callback function to develop more complex video applications. Therefore, this solution does not require dedicated hardware devices for video playback (most video capture card drivers support the VFW interface, it can meet the needs of video application development. Fortunately, VFW-related components are installed in today's Windows versions, and VC ++ has supported VFW since 4.0, which greatly simplifies the development of video applications. Currently, most of the video parts of PC-based multimedia applications are developed using VFW APIs.

VFW allows you to access video devices in message-driven mode, so that developers can easily control the data flow of devices. In short, this framework mainly includes VICAP. DLL, MSVIDEO. DLL, MCIAVI. DRV, AVIFILE. DLL, ICM, ACM, and other dynamic connection libraries. These components work together to capture, compress, and play videos. For more information about these modules, see MSDN.

(1) Video Capture

Video data is collected in real time through messages, macro functions, structures, and callback functions in the AVICAP module. The video capture process is as follows:

(1) create a capture window

The capCreateCaptureWindow () function is used to create a video capture window, which is the basis for all the capturing work and settings. Its main functions include: ① dynamically connect to or disconnect from the video and audio input devices; ② set the video capture rate; ③ provide the video source, video format, and whether to use the video compression dialog box; ④ set the display mode of video collection to Overlay or Preview; ⑤ obtain the video data of each frame in real time; ⑥ capture a video stream and audio stream and save it to an AVI file; 7. capture the data of a digital video frame and save the image of a single frame in DIB format. Users specify the file name of the captured data and copy the captured content to another file.

(2) Registration callback function

The register callback function is used to implement some special needs of users. In some real-time monitoring systems or video conferencing systems, data streams must be processed before being written to the disk for real-time effect. Applications can register callback functions in the capture window to handle the following situations in a timely manner: capture window status changes, errors, use video or audio cache, and discard control, the corresponding callback functions are capStatusCallback (), capErrorCallback (), capVideoStreamCallback (), capWaveStreamCallback (), and capYieldCallback ().

(3) Get the default settings of the capture window

Macro capturegetsetup (hWndCap, & m_Parms, sizeof (m_Parms.

(4) set parameters for the capture window

Macro capturesetsetup (hWndCap, & m_Parms, sizeof (m_Parms.

(5) connect the capture window and video capture card

Macro capDriveConnect (hWndCap, 0.

(6) obtain the functions and status of the Collection Device

You can use macro capdrivergetcaps (hwndcap, & m_capdrvcap, sizeof (capdrivercaps) to obtain the capabilities of video devices. You can use macro capgetstatus (hwndcap, & m_capstatus, sizeof (m_capstatus )) to obtain the status of the Video device.

(7) set the display mode of the capture window

The video is displayed in overlay and preview modes. In the overlay mode, video data is captured to deploy system resources, and the display speed is fast. The video collection format is YUV. You can set it through capoverlay (hwndcap, true; in Preview mode, system resources are occupied. The system calls the GDI function to display the video in the capture window. The video is displayed slowly. It supports the RGB video format.

(8) capture images to cache or files and process them accordingly

To process the collected data in real time, the callback mechanism should be used to capture a single frame of video by capsetcallbackonframe (hwndcap, framecallbackproc), and capsetcallbackonvideostream (hwndcap, videocallbackproc. If you want to save the collected data, you can call capcapturesequence (hwnd); to specify the file name, you can call capfilesetcapture (hwnd, filename ).

(9) terminate video capture. disconnect from the video capture device.

Call capcaturestop (hwndcap) to stop collection and call capdriverdisconnect (hwndcap) to disconnect the video window from the capture driver.

Since these APIs are closely related, we simply package them into a video capture class videocapture for ease of use.

The following code snippet shows the usage of this class:

// Create a video capture instance

Vidcap = new videocapture ();

// When frame capture is complete, the following sentence is used to call the display function of the Main Dialog Box class.

Vidcap-> SetDialog (this );

// Initialize the next line: connect to the driver and set the video format.

// Returns TRUE if the video capture device is successfully connected.

Vidcap-> Initialize ();

// If the connection is successful, we can get the BITMAPINFO related to the video format.

// Structure. The captured frames will be displayed later.

This-> m_bmp info = & vidcap-> m_bmp Info;

// Now, you can start video capture ......

Vidcap-> StartCapture ();

// Once the capture starts, the captured frame will arrive at the OnCaptureVideo function of the VideoCapture class.

// In this callback function, you can call the display function to display frames (see the next section)

// Stop capture

Vidcap-> StopCapture ();

// Release the video capture class after successful capturing.

Vidcap-> Destroy ();

[Note] To compile and link smoothly, you need to add the following statement before the class implementation file (VideoCapture. cpp:

# Pragma comment (lib, "vfw32 ")

# Pragma comment (lib, "winmm ")

(2) display captured video frames

There are many solutions for displaying captured video frames (that is, displaying images. For example, you can use SetDIBitsToDevice () to directly display captured video frames. However, this solution is very slow because it is a function based on the graphic device interface (GDI. In contrast, the better way is to use the DrawDib API to draw frames, because this function can be directly written to the video memory, so it can provide better performance.

The following code snippet shows how to use the DrawDib function to display captured video frames:

// Initialize DIB for drawing

HDRAWDIB hdib =: DrawDibOpen ();

// Then, use the appropriate parameter to call this function ......

: DrawDibBegin (hdib ,...);

// Now, you are ready.-You can call this function to display frames.

: DrawDibDraw (hdib ,...);

// Final, end frame painting

: DrawDibEnd (hdib );

: DrawDibClose (hdib );

In fact, the above code is very similar to the normal bitmap rendering process.

4. select an appropriate encoding/decoding library

In this article, we choose Roalt Aalmoes's open-source fast H.263 encoder library.

(1) sample code using Encoder

// Initialize the compressed object

CParam cparams;

Cparams. format = CPARAM_QCIF;

InitH263Encoder (& cparams );

// If you need to convert from RGB24 to YUV420 format, you should call the following function

InitLookupTable ();

// Create a callback function

// OwnWriteFunction is the global function called when encoding data is returned during encoding.

WriteByteFunction = OwnWriteFunction;

// The compressed data must be in YUV420 format.

// Call the following method before compression

ConvertRGB2YUV (IMAGE_WIDTH, IMAGE_HEIGHT, data, yuv );

// Compress the frame ......

Cparams. format = CPARAM_QCIF;

Cparams. inter = CPARAM_INTRA;

Cparams. Q_intra = 8;

Cparams. data = yuv; // the data format is YUV.

CompressFrame (& cparams, & bits );

// You can obtain compressed data from the callback function you have registered at the beginning.

// End the encoder.

// ExitH263Encoder ();

(2) decoder Programming

Note that the original H.263 encoder library is encoded in C mode and provides more details. In this article, we rewrite it with C ++.

The following is a sample code framework for decoder use:

// Initialize the decoder

InitH263Decoder ();

// Extract the frame ......

// Rgbdata must be large enough to store output data;

// The decoder generates image data in YUV420 format;

// After decoding, convert it to RGB24 format ......

DecompressFrame (data, size, rgbdata, buffersize );

// Terminate the decoder in the last step.

ExitH263Decoder ();

5. Run the application

To test the sample application in this article, you should copy the executable file to two different machines in a LAN, and then run them separately. Select the "connection" menu item from one machine, enter the name or IP address of the other machine in the pop-up dialog box, and click "Connect. In this case, a "Accept/reject" dialog box should pop up on another machine and click "accept. Then, the notification dialog box is displayed on the first machine. Click "OK" to start your video meeting (chat ......) .

Vi. Summary

I want to emphasize the following issues to readers through this article: audio and video development on Windows platforms is not so complicated and advanced; with the foundation of this article, you can also develop your own video conferencing, real-time monitoring system, video chat, and other software as needed. In addition, the technology introduced in this article can be modified and applied to Internet platforms.

At the same time, we also see that Microsoft's digital Video processing software development kit Video for Windows is indeed a great help for us, and with the help of open source multimedia packages on the internet can accelerate the development of such software faster.