WebRTC, a name derived from the abbreviation of Web
real-time communication ( English:Web Real-time communication), is an API that supports Web browsers for real-time voice conversations or video conversations. It was open source on June 1, 2011 and was included in the World Wide Web Consortium's recommended standard for Google, Mozilla and opera support [
1] [2] [3]. Http://baike.baidu.com/link?url=G9wblLo409MIqXQW1XDplFtdKgyol5_LXG8N4cxSYQzXuqc1blHymJrSzzdxRoPbWA37X_MYLNpZucZe0H1NjqIntroduction to Architectural components
(1) Your Web AppWeb developers develop programs that enable Web developers to develop real-time communication applications based on video and audio based on Web APIs provided by the integrated WebRTC browser. [1]
(2) Web APIThe WEBRTC standard API (Javascript) for third-party developers makes it easy for developers to develop Web applications similar to network video chats, and the latest standardization process can be viewed here. [2] These APIs can be divided into the network Stream API, Rtcpeerconnection, Peer-to-peer Data API three categories, detailed API description can be seen here. Network stream Apimediastream:mediastream is used to represent a stream of media data. Mediastreamtrack represents a media source in the browser. Rtcpeerconnectionrtcpeerconnection: A Rtcpeerconnection object allows users to communicate directly between two browsers. Rtcicecandidate: Represents a candidate for an ice protocol. Rtciceserver: Represents an ice Server. Peer-to-peer data apidatachannel: The Datachannel interface represents a bidirectional data channel between two nodes.
(3) WebRTC Native C + + APIThe local C + + API layer makes it easy for browser vendors to implement the WEBRTC standard Web API to process digital signal processes in an abstract way.
(4) transport/sessionThe transport/Session Layer Session layer component is implemented using a partial component of the Libjingle library without using Xmpp/jingle protocol a. RTP Stack protocol stack real time Protocolb. Stun/ice can establish call connections between different types of networks through STUN and ice components. C. Session management An abstract session layer that provides the ability to create and manage sessions. This layer protocol is left to the application developer to customize the implementation.
(5)VoiceengineThe audio engine is a framework that includes a range of audio multimedia processing, ranging from video capture cards to network-based transmission solutions. Ps:voiceengine is one of WEBRTC's most valuable technologies and is open source for Google's acquisition of Gips company. On VoIP, the technology industry is leading, and later articles will learn more about a. The isacinternet Speech Audio codec wideband and ultra-wideband audio codec for VoIP and audio streaming, is the default codec sampling frequency for the WEBRTC sound engine: 16khz,24khz,32khz ; (default is 16khz) adaptive rate is 10kbit/s ~ 52kbit/; adaptive packet size: 30~60ms; algorithm delay: Frame + 3msb.ilbcinternet Low bitrate Codecvoip Audio Stream narrowband Voice codec sampling frequency: 8khz;20ms frame bit rate is 15.2kbps30ms frame bitrate is 13.33kbps standard defined by IETF RFC3951 and RFC3952 C.neteq for Voice signal processing component NETEQ algorithm for audio software: Adaptive jitter control algorithm and voice packet loss concealment algorithm. Enables fast and high resolution adaptation to changing network environments, ensuring a graceful sound quality with minimal buffering latency. Gips Company is a cavatina technology that can effectively handle the impact of voice quality due to network jitter and voice packet loss. Ps:neteq is also a very valuable technology in WEBRTC, which has obvious effect on improving the quality of VoIP, and it is better to integrate the AEC\NR\AGC modules. The D.acoustic Echo canceler (AEC) echo Canceller is a software-based signal processing element that removes the echoes collected from the mic in real time. E.noise Reduction (NR) noise suppression is also a software-based signal processing element that is used to eliminate certain types of background noise (hiss, fan noise, etc.) associated with VoIP. ）
(6)VideoengineThe WEBRTC video processing engine Videoengine is a comprehensive framework that includes a range of video processing, from camera capture video to video information network transmission to video display throughout the complete process of the solution. A. VP8 video image codec is the default codec for the WEBRTC video engine VP8 suitable for real-time communication applications, because it is mainly a codec designed for low latency. The PS:VPX codec is open source after Google acquired ON2, VPX is now part of the WEBM project, and the WEBM project is one of the HTML5 standards that Google is committed to promoting. Video jitter buffer, which reduces the negative effects of video jitter and video packet loss. C. Image enhancements image quality Enhancement module handles images captured by the webcam, including shading, color enhancement, noise reduction, and other functions to improve video quality.VideoWEBRTC's video section contains functions such as acquisition, Codec (I420/VP8), encryption, media files, image processing, display, network transmission and flow control (RTP/RTCP). Video Capture---video_capture source code in the Webrtc\modules\video_capture\main directory, including the interface and the source code of each platform. On the Windows platform, WEBRTC uses DShow technology to capture device information and video data from enumerated videos, which means that most video capture devices can be supported, and for video capture cards (such as Hoi Hong HD cards) that require a separate driver. Video capture supports a variety of media types, such as I420, YUY2, RGB, Uyuy, etc., and can be frame size and frame rate control.
Video Codec---video_codingThe source code is in the Webrtc\modules\video_coding directory. WEBRTC uses I420/VP8 codec technology. VP8 is an open source implementation of Google's acquisition of ON2 and is also used in WEBM projects. VP8 can provide higher quality video with less data, especially for video conferencing needs.
Video Encryption--video_engine_encryptionVideo encryption is a part of the video_engine of WEBRTC, which is equivalent to the function of video application level, which provides the security of data on both sides of the point-to-point video to prevent the leakage of video data on the web. Video encryption on the sending side and the receiving side to decrypt video data, the key is negotiated between the two sides, the price will affect the performance of video data processing, or do not use video encryption, which will be better performance. The video-encrypted data source may be the original data stream, or it may be the encoded data stream. It is estimated that the encoded data stream will be less expensive and requires further research.
Video media files--media_fileThe source code is in the Webrtc\modules\media_file directory. This feature can be used as a video source with local files, a bit like the function of a virtual camera, and the supported formats are AVI. In addition, WEBRTC can also record audio and video to local files, more useful functions.
VideoImage processing--video_processing source code in the Webrtc\modules\video_processing directory. Video image processing for each frame of the image processing, including shading detection, color enhancement, noise reduction processing and other functions to improve video quality.
Video Display--video_renderThe source code is in the Webrtc\modules\video_render directory. On the Windows platform, WEBRTC uses Direct3D9 and DirectDraw to display video, only this way, it must.
network transmission and flow controlFor network video, the transmission and control of data is the core value. WEBRTC uses a mature rtp/rtcp technology.AudioThe audio portion of the WEBRTC contains features such as devices, codecs (ILIBC/ISAC/G722/PCM16/RED/AVT, Neteq), encryption, sound files, sound processing, sound output, volume control, audio and video synchronization, network transmission and flow control (RTP/RTCP).
Audio Equipment---audio_deviceSource code in the Webrtc\modules\audio_device\main directory, contains the interface and the source code of each platform. On the Windows platform, WEBRTC uses Windows Core Audio and Windows Wave technology to manage audio devices and also provides a mix manager. Audio output, volume control, and other functions can be achieved with the use of sound devices.
Audio Codec---audio_codingThe source code is in the Webrtc\modules\audio_coding directory. WEBRTC uses ILIBC/ISAC/G722/PCM16/RED/AVT codec technology. The WEBRTC also offers a NETEQ function---jitter buffers and packet loss compensation modules to improve sound quality and minimize latency. Another core feature is the voice conferencing-based mixing process.
Sound Encryption--voice_engine_encryptionAs with video, WEBRTC also provides sound encryption capabilities.
Sound FilesThis feature is available with local files as an audio source, supported in the format of PCM and WAV. Similarly, WEBRTC can record audio to a local file.
Sound processing--audio_processingThe source code is in the Webrtc\modules\audio_processing directory. Sound processing is processed for audio data, including functions such as echo cancellation (AEC), AECM (AEC Mobile), Automatic gain (AGC), noise Reduction (NS), mute detection (VAD) processing, to improve sound quality.
network transmission and flow controlLike video, WEBRTC uses mature rtp/rtcp technology.
Introduction to "WebRTC"
