"Reprint" WEBRTC Basic introduction

"WebRTC, a name derived from the abbreviation of Web real-time communication (Web real-time communication), is a technology that supports web browsers for real-time voice conversations or video conversations, and is Google's $68.2 million acquisition of global IP in 2010 Solutions company to get a technology, Google Open source of this technology! ”

Just for the more than 60 million dollar technology, it is worth studying! WEBRTC provides video conferencing core technology, including audio and video capture, codec, network transmission, display and other functions, and also support cross-platform: windows,linux,mac,android, especially in real-time communication technology can be flexibly grafted into non-RTC class streaming media system , for example, video jitter control at low bandwidth, image enhancement, universal NAT penetration, and so on (let's elaborate on how to graft later), so this requires familiarity and control of WEBRTC's overall process and deep-seated framework, We first understand the WebRTC browser's basic interface and the process of a realtime video call through the browser-side WEBRTC Video Call process.

WEBRTC structure

First of all, from the WEB-RTC aspect analysis WEBRTC on the browser interface structure, in the browser side, WEBRTC mainly implemented three interfaces

• MediaStream, to achieve the encapsulation of local audio and video resources, such as from camera, microphone, remote stream and so on, MediaStream represents a media data stream, a Mediastreamtrack represents a media source in the MediaStream, such as audio, video, subtitles, and so on.
• Rtcpeerconnection, voice or video call process, the internal cover call, answer, penetrate, encrypt, transmit and session management a series of processes, a rtcpeerconnection represents one end of a pair of call process.
• Rtcdatachannel, on the peerconnection, transmits the custom data.

#MediaStream

On the browser side, the MediaStream interface name is Getusermedia, which provides simultaneous audio and video streaming to the upper layer, such as when local media resources are available, MediaStream can be a local camera video track and a local The audio track provided by microphone is synchronized with the stream. Of course, after the browser to obtain audio and video data, you can do a variety of localized processing, such as capture, image style changes, local display filters and so on.

#RTCPeerConnection

The rtcpeerconnection is primarily designed to handle connection and data transfer between point-to-point, making the process stable and efficient. Under the rtcpeerconnection, a large number of codec, communication protocol work to achieve the entire real-time communication process, even in the case of stable bandwidth can not provide real-time communication, the main function points include:

• Packet loss compensation (packet loss concealment)
• Echo Cancellation (echo cancellation)
• Adaptive bandwidth (bandwidth adaptivity)
• Video jitter buffer (dynamic jitter buffering)
• AGC (Automatic gain control)
• Noise reduction and suppression (Noise Reduction and suppression)
• Image cleanup (images ' cleaning ') (!)

#RTCDataChannel

Rtcdatachannel provides a way to exchange custom data on top of rtcpeerconnection, transferring custom data on peerconnection compared to streaming data, not only in volume, but also in terms of reliability, security, flexibility, far Enough to meet the needs. This provides greater convenience in the development of audio-video-based games and applications.

WEBRTC components

#Transport/session

• Complete RTP/SRTP protocol stack
• STUN, TURN, ice process
• Session management mechanism.

#VoiceEngine

WEBRTC is one of the most valuable technology, support 722,PCM,ILBC,ISAC and other coding, on the VoIP, the technology industry leader!

Neteq algorithm: Adaptive jitter Control algorithm and speech packet loss concealment algorithm. Enables fast and high resolution adaptation to changing network environments, ensuring a graceful sound quality with minimal buffering latency. Can effectively deal with the impact of voice quality due to network jitter and voice packet loss. Neteq is also a very valuable technology in WEBRTC, it has obvious effect on improving VoIP quality, and it is better to integrate the AEC (echo cancellation) \nr (noise suppression) \AGC and other modules.

#VideoEngine

The VP8 video image codec, the default codec for the WEBRTC video engine, is VP8 suitable for real-time communication applications because it is primarily designed for low-latency codecs.

Video jitter buffers can reduce the negative effects of video jitter and video packet loss.

The image quality Enhancement module handles images captured by the webcam, including shading detection, color enhancement, noise reduction, and other functions to improve video quality.

"Reprint" WEBRTC Basic introduction