WEBRTC Introduction and simple Application

WEBRTC Introduction and simple Application

WebRTC, web Real-time communication, Web real-time communication technology. In short, the introduction of real-time communication in a Web browser, including audio and video calls.

• WEBRTC Real-time communication technology Introduction
• How to use
• Media Introduction
• Signaling
• Stun and turn Introduction
• Peering connections and offer/reply negotiation
• Data channel
• Nat and Firewall penetration
• Simple Application
• Other

WEBRTC Real-time communication technology Introduction

WEBRTC implements a web-based voice conversation or video call with the goal of no plug-in for real-time communication on the Web.

WEBRTC provides the core technology of video conferencing, including audio and video collection, codec, network transmission, display and other functions, and also support cross-platform, including Linux, Windows, Mac, Android and so on.

1. WebRTC Triangle

2. WEBRTC trapezoid

3. WEBRTC Multi-party sessions

WEBRTC supports multi-party sessions or conferencing sessions with multiple browsers, there are two modes to establish such sessions:

4. New WEBRTC Features

How to use WEBRTC

WEBRTC is easy to use and requires minimal steps to establish a media session. Some messages flow between the browser and the server, while others flow directly between the two browsers (which become peers).

1. Establish WEBRTC session

There are several steps required to establish a WEBRTC connection:

• Get local media ( getUserMedia() ,mediastream API)
• Establishing a peering connection between the browser and peer (other browser or terminal) (rtcpeerconnection API)
• Associating media and data channels to the connection
• Swap Session description (rtcsessiondescription)

• Browser m requests Web page from Web server
• Web server returns a Web page with WebRTC JS to M
• Browser l request Web page from Web server
• Web server returns a Web page with WebRTC JS to L
• M decides to communicate with L by sending M's session description object (offer, proposal) to the Web server through M's own JS
• The Web server sends the session description object of M to JS on l
• JS on L sends the session description object (answer, answer) of L to the Web server
• The Web server forwards the JS on the answer to M
• M and L start interacting to determine the best way to access each other
• After completion, M and L begin negotiating the communication key
• M and L start exchanging voice, video, or data

WEBRTC triangle Session Specific invocation flow:

说明：    SDP对象的传输可能是一个来回反复的过程，并且该过程采用的协议并未标准化

WEBRTC trapezoid session Mode specific invocation process:

说明：    此场景中，浏览器M和L直接交换媒体，只是它们运行的Web服务器不用而已。每个浏览器的会话描述对象都会映射至Jingle[XEP-0166]session-initiate消息和session-accept方法。    

Media Introduction

First look at the local media in WEBRTC:

1. Media in WEBRTC

• Track (Mediastreamtrack, which represents a single type of media that a device or recording can return, uniquely associating a "source", WEBRTC cannot directly access or control "source", and all control over "source" is implemented through the track; a "source" May correspond to multiple track objects)
• Stream (MediaStream, a collection of track objects)

The orbits and streams are illustrated as follows:

2. Capturing local media

The following code shows a simple acquisition of local media and shows:

//Note the difference between Getusermedia () in each browser//Opera --Getusermedia//Chrome-Webkitgetusermedia//Firefox --MozgetusermediaNavigator.Getusermedia = Navigator.Getusermedia || Navigator.Webkitgetusermedia || Navigator.Mozgetusermedia;    //Get video only:varConstraints= {Audio: false, Video: true};  varVideo= Document.Queryselector("Video");    function Successcallback(stream){      //Note:make The returned stream available to console for inspection    window.Stream =Stream;        if(window.URL){          //Chrome browser        Video.Srcobject =Stream;      } Else {          //Firefox and Opera: You can set the video source to stream directly        Video.src =Stream;      }      //Play    Video.Play();  }    function Errorcallback(Error){      Console.Log("Navigator.getusermedia Error:",Error;  }    Navigator.Getusermedia(Constraints,Successcallback,Errorcallback);

The results are as follows:

Complete code view: Https://github.com/caiya/webrtc-demo.git

Signaling

In the WEBRTC, signaling plays a pivotal role. But implementations are not standardized, such as HTTP, WebSocket, XMPP, and so on.

1. The role of signaling

• Negotiating media features and settings
• Identify and verify the identity of the session participant (Exchange information in the SDP object: media type, codec, bandwidth, and so on)
• Controlling media sessions, indicating progress, changing sessions, terminating sessions
• Double occupancy decomposition

Simply put, signaling is the process of coordinating communications, and once the signaling service is established, a connection is established between the two clients, in theory they can communicate in a point-to-point manner.

2. Transmission of signaling

WEBRTC requires a two-way signaling channel to be established on two peers, typically three ways to transmit WEBRTC signaling:http, websocket, data channel

The HTTP method is as follows:

WebSocket Proxy Signaling Transmission:

3. Servers in WEBRTC

WEBRTC provides browser-side communication, but does not mean that WEBRTC does not require a server. Aside from the application server, at least the following two types of servers are required:

• A server (signaling service) that exchanges various meta-data (signaling) between browsers before communication is made
• Servers that traverse NAT and firewalls (stun, turn, RSIP, etc.)

说明：    元数据是通过信令服务器中转发给另一个客户端，但是对于流媒体数据，一旦会话建立，首先尝试使用点对点连接。简单一点说就是：每个客户端都有一个唯一的地址，他能用来和其他客户端进行通讯和数据交换。        STUN服务器：用来取外网地址的。（见下节）        TURN服务器：在P2P失败时进行转发的。（见下节）        ICE：*Interactive Connectivity Establishment*，即交互式连通建立方式。并非一种新的协议，它通过综合利用现有NAT穿透协议，以一种更有效的方式来组织会话建立过程，使之在不增加任何延迟同时比STUN等单一协议更具有健壮性、灵活性。

4, signaling interaction and the establishment of rtcpeerconnection

WEBRTC uses rtcpeerconnection to establish a connection transfer stream, two things need to be done after the Rtcpeerconnection instance is established to establish a point-to-point channel:

• Determine the characteristics of the media stream on this computer, such as resolution, codec capability (SDP descriptor)
• Network address of the host on both ends of the connection (ICE candidate)

Exchange SDP descriptors via offer and answer:

• A and B each build a PC instance
• A offer signaling with an SDP descriptor containing a is established via the Createoffer () method provided by the PC
• A setlocaldescription () method provided by the PC to give the SDP descriptor of A to a PC instance
• A will send the offer letter to B via the server
• b extracts the SDP descriptor contained in the offer signaling of a, and gives the PC instance of B via the Setremotedescription () method provided by the PC
• b a Createanswer () method provided by the PC to create an SDP descriptor answer signaling with B
• b by the Setlocaldescription () method provided by the PC, the SDP descriptor of B is given to the PC instance of B
• b Send answer signaling via server to a
• After receiving the answer signaling of B, extract the SDP descriptor of B and call the Setremotedescripttion () method to the PC instance

Establish a nat/firewall traversal connection via the ice framework:

WEBRTC uses the ice framework to obtain an address that can be accessed directly by the outside world, and rtcpeerconnection can pass in the address of the ice server when it is created, such as:

var={    "iceServers": [{        "url":"stun:stun.l.google.com:19302"    }]};var=newRTCPeerConnection(iceServer);

• A, b each creates a PC instance configured with an ice server and adds a Onicecandidate event callback to it
• When a network candidate is available, the Onicecandidate function is called
• Inside the callback function, a or B encapsulates the network candidate's message in an ice candidate signaling, which is routed through the server to the other party
• When a or B receives an ice candidate signaling sent by the other party via the server, it resolves and obtains the network candidate and adds it to the PC instance via the PC instance's Addicecandidate () method.

This connection is completed and the stream data can be transferred to the rtcpeerconnection via Addstream ().

Stun and turn Introduction

The browser is a very popular design after the network address translation device (NAT). Give me a chestnut:

Take a look at the "public address" and "Private Address" below:

NAT is primarily responsible for maintaining a mapping table between the internal IP address and port number and the external IP address and port number.

1. Stun server

STUN,session traversal Utilities for Nat, called NAT session Traversal utility server. To put it simply, you get the outermost NAT (public IP address) information for your intranet device.

2. Turn server

TURN,traversal Using Relay around Nat, called a repeater NAT traversal server.

说明：    媒体中继地址是一个公共地址，用于转发接收到的包，或者将收到的数据包转发给浏览器。如果两个对等端因为NAT类型等原因不能直接建立P2P连接的话，那么可以使用中继地址。        ps：相比较直接使用web服务器提供媒体中继理想点。

Peering connections and offer/reply negotiation

The previous section briefly describes the peering and offer/answer interaction processes, as explained in this section.

In fact, WEBRTC defines two main sets of functions, namely: Media Capture (Getusermedia (), previously described), media transfer. The concept of peer connection and offer/reply negotiation is the core of media transmission.

1. Peering connection

The rtcpeerconnection interface is the primary API for WEBRTC, which is used to establish media connections and data connection paths on the peer side. The constructor of the rtcpeerconnection object has a series of properties, most notably the Iceservers property, which represents the server address list. Used to help establish sessions through NAT and firewalls.

varPc= New rtcpeerconnection({    iceservers:[{        URL: ' stun:stun.l.google.com:19302 '    },{        URL: ' turn:[email protected] ',        Credential: ' Test '    }]})Getusermedia({    Audio: true,    Video: true},Successcb,FAILURECB)function SUCCESSCB(stream){    //Tell the browser that I want to send MediaStream    PC.Addstream(stream)//Removestream ()}

2. Proposal/Response Negotiation

To establish a connection between the two, a session must be established between the two. Offer/answer is a "one-pass" type of negotiation mechanism. The process may be repeated several times in practice.

WEBRTC uses rtcsessiondescription objects to represent proposals and answers. Each browser will generate one of the objects.

3. JavaScript Proposal/Response negotiation control

The local browser only focuses on two specific calls:

// 将我的会话描述告知我的浏览器pc.setLocalDescription(mySessionDescription)...// 将对等端的会话描述告知我的浏览器pc.setRemoteDescription(yourSessionDescription)

Generate a proposal, answer:

//Generate proposalsPC.Createoffer(Gotoffer,Didntgetoffer)function Gotoffer(asessiondescription){    setlocaldescription(asessiondescription) ...the session description (proposed offer) can now be sent to the peer so that the peer    //a), transmit the proposal to setremotedescription    //b), call Createanswer}//Generate answerPC.Createanswer(Gotanswer,Didntgetanswer)function Gotanswer(asessiondescription){    setlocaldescription(asessiondescription) ...///The session description (answer answer) is now sent to the peer so that the peer    //a), pass the response to Setremotedescription}

4. Test Demo Instructions

The following test demo shows a live video call in two browsers, source address: Https://github.com/caiya/webrtc-p2p.git

Data channel

Rtcdatachannel, a data channel is a non-media interactive connection established between browsers. That is, the media message is not delivered and the data is passed directly around the server. The data channel supports large traffic and low latency compared to websocket and HTTP messages.

注意：    单个对等连接中的多个数据通道底层共享一个流，所以只需一次offer、answer即可建立首个数据通道。之后再建立数据通道无需再次进行offer、answer交换。        典型应用：游戏实时状态更新。    

Use of data channels

You can create a data channel only after you have created the rtcpeerconnection instance, as follows:

=newRTCPeerConnection=pc.createDataChannel('')

Once the data channel is created at one end, the other end only needs to listen for the ondatachannel event:

=newRTCPeerConnection()pc.ondatachannel=function{    =e.channel}

At this point, two peers have established a data channel with each other and can send messages to each other directly:

DC.Send(' I am a text string for sending ')DC.Send(New Blob([' I am a Blob object '], {type: ' Text/plain '}))DC.Send(New ArrayBuffer( +))//Send ArraybufferDC.OnMessage = function(e){    Console.Log(' received message: ', e.Data)}

Test demo After joining the data channel

Project source code address: Https://github.com/caiya/webrtc-p2p-datachannel

Part:

Author @ Zhongzhou
November 27, 2017

WEBRTC Introduction and simple Application