Home > Developer > Web Develop
It is well known that the browser itself does not support the direct establishment of communication channels between each other through the server to relay. For example, there are now two clients, A and B, they want to communicate, first need a and server, B and server to establish a channel between. A to send a message to B, a first send the message to the server, the server to a message to relay, sent to B, the reverse is the same. So a message between A and b through two channels, the efficiency of communication is subject to the bandwidth of both channels. At the same time such a channel is not suitable for the transmission of data flow, how to establish a point-to-point transmission between browsers, has plagued the developers. WebRTC was born
WEBRTC is an open source project designed to enable browsers to provide a simple JavaScript interface for real-time communication (RTC). The simple and clear point is to let the browser provide JS instant communication interface. The channel created by this interface is not the same as websocket, the communication between a browser and the WebSocket server, but through a series of signaling, to establish a browser and browser (Peer-to-peer) channel, the channel can send any data, Without having to go through the server. and WEBRTC through the implementation of MediaStream, through the browser to invoke the device's camera, microphone, so that the browser can transfer audio and video
WEBRTC is already in our browser.Such a good function, the major browser manufacturers will naturally not ignore. Now WEBRTC can already be used in newer versions of Chrome, opera and Firefox, and a well-known browser compatibility query site Caniuse provides a detailed browser compatibility
In addition, according to 36Kr ago the news Google launched the support WEBRTC and web audio Android version of Chrome 29@36kr and Android version of opera began to support WEBRTC, Allows users to implement voice and video chat without any plugins, Android also starts to support WEBRTC
Three interfacesThe WEBRTC implements three APIs, namely:
* MediaStream: Through the MediaStream API can get video, audio synchronization flow through the device's camera and microphone
* Rtcpeerconnection:rtcpeerconnection is a component that WEBRTC uses to build stable, efficient streaming between point-to-point
* Rtcdatachannel:rtcdatachannel enables a high-throughput, low-latency channel between browsers (Point-to-point) for transmitting arbitrary data
Here's a general introduction to these three APIs
MediaStream (Getusermedia)The MediaStream API provides WEBRTC with the ability to get video, audio streaming data from the device's camera, microphone
StandardStandard Portal
How to InvokeThe same can be done by calling Navigator.getusermedia (), which accepts three parameters:
1. A Constraint object (Constraints objects), which will be followed by a separate
2. A call to a successful callback function, if the call succeeds, pass it to a stream object
3. A call failed callback function, if the call fails, pass it an Error object
Because browser implementations are different, they often prefix the method before implementing the standard version, so a compatible version is like this
A super-simple examplevar getusermedia = (Navigator.getusermedia | | Navigator.webkitgetusermedia | | Navigator.mozgetusermedia | | Navigator.msgetusermedia);
Here is a super simple example to show the effect of Getusermedia:
<!doctype html> <html lang="ZH-CN"> <head> <meta charset="UTF-8"> <title>Getusermedia instances</title> </head> <body> <video id="video" autoplay></video> </body> <script type="Text/javascript"> varGetusermedia = (Navigator.getusermedia | | navigator.webkitgetusermedia | | navigator.mozgetusermedia | | Navigator.msgetusermedia); Getusermedia.call (Navigator, {video:true, Audio:true}, function(localmediastream) {varVideo =Document. getElementById (' video '); VIDEO.SRC =window. Url.createobjecturl (Localmediastream); Video.onloadedmetadata = function(e) {Console. log ("Label:"+ Localmediastream.label);Console. log ("Audiotracks", Localmediastream.getaudiotracks ());Console. log ("Videotracks", Localmediastream.getvideotracks ()); }; }, function(e) {Console. log (' reeeejected! ', e); });</script> </html>
Save this piece of content in an HTML file and place it on the server. With a newer version of opera, Firefox, Chrome Open, in the browser pop-up asking whether to allow access to the camera and the microphone, choose to agree, the browser will appear on the camera on the screen.
Note that the HTML file to be placed on the server, otherwise you will get a navigatorusermediaerror error, display Permissiondeniederror, the simplest way is the CD to the HTML file in the same directory, and then python-m Simplehttpserver (loaded with Python), then enter the http://localhost:8000/{file name in the browser}.html
After using Getusermedia to get the stream, you need to output it, typically binding to the output on the Video tab, and you need to use window. Url.createobjecturl (Localmediastream) to create a blob URL that can be played in video using the SRC attribute, notice that the AutoPlay attribute is added to the video, or only one picture can be captured
Once the stream has been created, the Label property can be used to obtain its unique identity, and the Getaudiotracks () and Getvideotracks () methods can be used to get an array of trace objects for the stream (if a stream is not turned on, its tracking object array will be an empty array)
Constraint Object (Constraints)Constraint objects can be set in the Addstream method of Getusermedia () and rtcpeerconnection, a constraint object that is used by WEBRTC to specify what kind of stream to accept, which can define the following properties:
* Video: Whether to accept the stream
* Audio: Whether to accept the stream
* MinWidth: Minimum width of the video stream
* MaxWidth: Maximum width of the video stream
* MinHeight: Minimum height of the video stream
* MAXHIEHGT: Maximum height of the video stream
* Minaspectratio: Minimum aspect ratio of the video stream
* Maxaspectratio: Maximum aspect ratio of the video stream
* Minframerate: Minimum frame rate for video streams
* MaxFrameRate: Maximum frame rate of the video stream
For details see resolution Constraints in Web Real time Communications draft-alvestrand-constraints-resolution-00
RtcpeerconnectionWEBRTC uses rtcpeerconnection to pass streaming data between browsers, which are point-to-point and do not need to be brokered by the server. But this does not mean that we can abandon the server, we still need it to pass signaling (signaling) to build this channel. WEBRTC there is no protocol defined for signaling to establish a channel: signaling is not part of the Rtcpeerconnection API
SignalingSince there is no protocol to define the specific signaling, we can choose any way (AJAX, WebSocket), use arbitrary protocol (SIP, XMPP) to pass the signaling, establish the channel, such as I write the demo, is the WS module of node, to pass the signaling on the WebSocket
There are three types of information that need to be exchanged by signaling:
* Session information: Used to initialize the communication and error
* Network configuration: such as IP address and port what
* Media Adaptation: What Encoders and resolutions can be accepted by the sender and receiver's browser
The exchange of this information should be completed before the point-to-point stream is transmitted, and a general architecture diagram is as follows:
Establishing a channel through a serverHere again, even if WEBRTC provides a point-to-point channel between browsers for data transmission, it is necessary for the server to be involved in establishing this channel. WEBRTC requires the server to support four aspects of its functionality:
1. User Discovery and communication
2. Signaling Transmission
3. nat/Firewall traversal
4. If the point-to-point communication fails to establish, it can be used as a transit server
A common problem in establishing a point-to-point channel is the NAT traversal technique. NAT traversal technology is required to establish a connection between hosts in a private TCP/IP network that uses a NAT device. This problem has often been encountered in the VoIP field in the past. There are already many NAT traversal techniques, but none is perfect, because NAT behavior is nonstandard. Most of these technologies use a common server, which uses an IP address that can be accessed from anywhere in the world. In Rtcpeeconnection, the ice framework is used to ensure that the rtcpeerconnection can achieve NAT traversal
ICE, called Interactive Connection creation (Interactive Connectivity establishment), is a comprehensive NAT traversal technology, a framework that integrates various NAT traversal techniques such as stun, TURN (traversal Traversal of the Using Relay NAT relay implementation). Ice will first use stun, try to establish a UDP-based connection, if it fails, it will go to TCP (try HTTP first, then try HTTPS), and if it fails, ice will use a trunk turn server.
We can use Google's stun server: stun:stun.l.google.com:19302, so an architecture that integrates the ice framework should look like this.
Browser compatibleor a different prefix, using a method similar to the one above:
Create and usevar peerconnection = (windowwindow window windows. mozrtcpeerconnection);
//Using Google's stun server varIceserver = {"Iceservers": [{"url":"stun:stun.l.google.com:19302"}]};//browser-compatible Getusermedia notation varGetusermedia = (Navigator.getusermedia | | Navigator.webkitgetusermedia | | Navigator.mozgetusermedia | | Navigator.msgetusermedia);//browser-compatible peerconnection notation varPeerconnection = (window. peerconnection | |window. webkitPeerConnection00 | |window. webkitrtcpeerconnection | |window. mozrtcpeerconnection);//WebSocket connection to the background server varSocket = __createwebsocketchannel ();//Create Peerconnection instances varPC =NewPeerconnection (Iceserver);//Send ice candidates to other clientsPc.onicecandidate = function(event) {Socket.send (JSON. stringify ({"Event":"__ice_candidate","Data": {"Candidate": Event.candidate});};//If the media is detected to be connected to the local, bind it to a video label on the outputPc.onaddstream = function(event) {somevideoelement.src = Url.createobjecturl (Event.stream);};//Get local media stream and bind to output on a video label, and send this media stream to other clientsGetusermedia.call (Navigator, {"Audio":true,"Video":true}, function(stream) {//Send offer and answer functions, send local session description varSENDOFFERFN = function(DESC) {pc.setlocaldescription (DESC); Socket.send (JSON. stringify ({"Event":"__offer","Data": {"SDP": desc})); }, SENDANSWERFN = function(DESC) {pc.setlocaldescription (DESC); Socket.send (JSON. stringify ({"Event":"__answer","Data": {"SDP": desc})); };//Bind the media stream to the video tag for outputMYSELFVIDEOELEMENT.SRC = Url.createobjecturl (stream);//Add a stream to the peerconnection that needs to be sentPc.addstream (stream);//If the sender sends an offer signaling, otherwise sends a answer signaling if(Iscaller) {Pc.createoffer (SENDOFFERFN); }Else{Pc.createanswer (SENDANSWERFN); }}, function(error) {//Processing Media stream creation failure error});//Handling Incoming signalingSocket.onmessage = function(event) {varJSON =JSON. Parse (Event.data);//If it is an ice candidate, add it to the peerconnection, otherwise set the session description as passed. if(Json.event = = ="__ice_candidate") {Pc.addicecandidate (NewRtcicecandidate (json.data.candidate)); }Else{Pc.setremotedescription (NewRtcsessiondescription (JSON.DATA.SDP)); }}; Instance
Because of the more complex and flexible signaling transmission, so there is no short example, you can go directly to the last
RtcdatachannelNow that you can set up a point-to-point channel to deliver real-time video and audio streams, why not use this channel to pass a bit of other data? The Rtcdatachannel API is used to do this, based on which we can transfer arbitrary data between browsers. Datachannel is built on peerconnection and cannot be used alone.
Using DatachannelWe can use Channel = Pc.createdatachannel ("Somelabel") to create a data Channel on an instance of peerconnection and give it a label
Datachannel is used in almost the same way as WebSocket, there are several events:
* OnOpen
* OnClose
* OnMessage
* OnError
At the same time it has several states that can be obtained by readystate:
* Connecting: Browser is trying to establish channel
* Open: Established successfully, you can send data using the Send method
* Closing: Browser is closing channel
* Closed:channel has been shut down.
Two methods of exposure:
* Close (): Used to close channel
* Send (): Used to send data to each other via channel
JavaScript has provided the file API to extract files from the elements of input[type= ' file '], and filereader to convert the files to Dataurl, This also means that we can divide the dataurl into multiple fragments to transfer files over the channel.
A comprehensive demoSkyrtc-demo, this is a demo I wrote. Set up a video chat room, and be able to broadcast files, of course, also support single-to-single file transfer, writing is also very rough, the latter will continue to improve
How to useVideo Audio Chat (connected to the camera and microphone, at least to have a webcam), broadcast files (can be transmitted separately, provide APIs, broadcast is based on the implementation of separate propagation, can be spread at the same time, small files okay said, large files waiting for memory to eat up), broadcast chat information
ResourcesUsing WEBRTC to build a front-end video chat room-introductory article
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
