Google open real-time communication framework WebRTC source code

In fact, as early as June 2 ago, a friend who worked at Google told me this information. I also got all the source code from WebRTC for the first time, but since my recent work was really busy, this information was not immediately reproduced here. Now, I want to keep an eye on the multimedia applications, learn the technologies in the WebRTC Framework earlier, and use them in actual projects.

Google today announced the opening of the WebRTC architecture to developersSource code.WebRTC is a technology that enables real-time video and audio communication within the browser. It was acquired by Google last year when it acquired global IT solutions for $68.2 million billion.

Google said in its official blog today: "We hope to make browsers an innovative place for real-time communication. So far, real-time communication requires the use of copyrighted signal processing technology, and can be implemented through plug-ins or download clients, while WebRTC allows developers to use HTML and JavaScript APIs to create real-time applications."
Google also said: "To this end, we will work closely with browser vendors such as Mozilla and opera to make the broader webCommunityTo deploy this technology. In addition, we will work with standards bodies such as IETF and W3C working groups to define a set of real-time communication standards ."

Currently, developers can access http://sites.google.com/site/webrtc/page to obtain the webrtcsource.CodeSpecifications and tools. (The network in China may not be able to log on. The total size is 80 mb. If necessary, leave the contact information)

Here I also reposted a brief introduction to WebRTC source code (http://blog.csdn.net/cymlife/archive/2011/06/02/6460852.aspx) components. If you want to learn more about related technologies, you must take the time to study. I will also study, and I will post relevant information in timeArticleCommunicate with you.

Video

The video part of WebRTC includes functions such as collection, encoding/decoding (i420/Vp8), encryption, media files, image processing, display, network transmission and traffic control (RTP/RTCP.

Video collection --- video_capture

The source code is in the WebRTC \ modules \ video_capture \ main directory and contains the interface and the source code of each platform.

On Windows, WebRTC uses dshow technology to collect device information and video data of video enumeration. This means that most video collection devices are supported.ProgramVideo Acquisition Card (such as haikang HD card) is powerless.

Video collection supports multiple media types, such as i420, yuy2, RGB, and uyuy, and supports frame size and frame rate control.

Video Codec --- video_coding

The source code is in the WebRTC \ modules \ video_coding directory.

WebRTC adopts i420/Vp8 encoding/decoding technology. Vp8 is an open-source implementation after Google acquired on2 and is also used in the webm project. Vp8 can provide high-quality videos with less data, which is especially suitable for video conferencing.

Video Encryption-video_engine_encryption

Video Encryption is part of WebRTC's video_engine, which is equivalent to the video application layer function. It provides data security for both peer-to-peer videos and prevents leakage of video data on the web.

Video Encryption encrypts and decrypts video data on the sender and receiver. The key is negotiated by both parties, at the cost of affecting the video data processing performance. You can also disable the video encryption function, this will improve the performance.

The video encryption data source may be the original data stream or the encoded data stream. It is estimated that the data stream is encoded, so the encryption cost will be lower and further research is required.

Video Media File-media_file

The source code is in the WebRTC \ modules \ media_file directory.

This function can use local files as video sources, which is similar to the functions of virtual cameras. The supported formats include Avi.

In addition, WebRTC can also be used to record audio and video files to local files.

Video Image Processing-video_processing

The source code is in the WebRTC \ modules \ video_processing directory.

Video Image Processing processes each frame of the image, including brightness detection, color enhancement, and noise reduction, to improve the video quality.

Video Display -- video_render

The source code is in the WebRTC \ modules \ video_render directory.

On Windows, WebRTC uses Direct3D9 and DirectDraw to display videos. This is only required.

Network Transmission and Traffic Control

For Network Videos, data transmission and control are core values. WebRTC adopts mature RTP/RTCP technology.

Audio

WebRTC audio part, including device, codec (ilibc/ISAC/g722/pcm16/red/AVT, neteq), encryption, sound file, sound processing, sound output, volume control, audio/video synchronization, network transmission and traffic control (RTP/RTCP) and other functions.

Audio Device --- audio_device

The source code is in the WebRTC \ modules \ audio_device \ main directory and contains the interface and the source code of each platform.

On Windows, WebRTC uses Windows core audio and Windows wave technologies to manage audio devices. It also provides a sound mixing manager.

Audio devices can be used to achieve sound output, volume control, and other functions.

Audio Codec --- audio_coding

The source code is in the WebRTC \ modules \ audio_coding directory.

WebRTC uses ilibc/ISAC/g722/pcm16/red/AVT encoding/decoding technology.

WebRTC also provides the neteq function-the jitter buffer and packet loss compensation module, which can improve the sound quality and minimize the latency.

Another core function is audio mixing processing based on voice conferencing.

Voice encryption -- voice_engine_encryption

Like videos, WebRTC also provides sound encryption.

Sound file

The local file can be used as the audio source. The supported formats include PCM and WAV.

Similarly, WebRTC can also record audio to a local file.

Audio Processing-audio_processing

The source code is in the WebRTC \ modules \ audio_processing directory.

Audio processing processes audio data, including Echo Cancellation (AEC), aecm, automatic gain (AGC), and noise reduction, to improve sound quality.

Network Transmission and Traffic Control

Like videos, WebRTC uses mature RTP/RTCP technologies.