From zero to one, develop a Zoom bar with the live audio and video SDK

Zoom (zoom.us) is a widely used online conferencing software. I believe you must be in office, conference, chat and other scenarios under the experience or use, as a mature business software, Zoom provides a stable real-time audio and video call quality, as well as whiteboard, chat, screen sharing, PPT screening and other common functions. But in today's browser becomes the mainstream of the era, real-time audio and video how willing to lag behind it? Developing a similar conferencing software directly on a Web page is certainly more of a concern than a Zoom that requires a package to be installed. When a meeting is needed, you can access and start the meeting directly through a link. Now, with the seven bull live audio and video Web SDK, we can easily turn your ideas into reality.

First, let's tease out what key points need to be compromised for a web-only online conferencing product

• Browser compatibility is good and requires support for most major desktop browsers.

The seven bull live Audio video is based on Google's WebRTC protocol on Chrome, which has been formally written into WEB standards and is well-compatible with all modern browsers.

• Better call quality, low latency, high definition

Unlike traditional WebRTC that communicate in the form of user-to-peer communication, we use nodes that are deployed globally as low-latency, real-time interactive networks to communicate with each other on all sides, guaranteeing latency while ensuring the quality of calls.

• Meeting function to be rich, PPT demonstration, whiteboard, screen sharing and so on I want to

Our SDK provides a rich list of features to meet the needs of most meeting scenarios, and in theory the SDK can be used to completely re-engrave a Web version of zoom.

• So much to say, is it difficult to access? Are there any examples and documentation?

Of course! Now it's time to experience our Web Demo (open in a desktop browser). Https://demo-rtc.qnsdk.com. Demo source code is also open source on Github for your reference Https://github.com/pili-engineering/QNRTC-Web

This demo realizes most of the functions directly provided by the SDK, the demo of integrated whiteboard/ppt sharing/chatting and so on is ready to go on the line, so please look forward to it. About the specific process of access we will briefly introduce the following, detailed instructions and references can move to our document station HTTPS://DEVELOPER.QINIU.COM/RTN/SDK/4412/DESCRIPTION-OF-WEB-SDK

Development process

A simple meeting product, typically through the following process:

• User Registration/Login (Developer's own integration, SDK only need to be used to differentiate user userid)
• Create a meeting room/join a conference Room
• Collect your own camera/microphone data
• Publish collected media data to a room
• Subscribe to other people's media data in the room and play it in real time
• Process user Join/leave, publish/Unpublish

This simplifies the various functions of the SDK, in fact, is to join the room-the acquisition of local media streaming-media streaming-subscription media streaming-event processing, the SDK for each step is a simple package, using a few lines of code can be done.

Introducing the SDK

We recommend using NPM to introduce our SDK directly npm i pili-rtc-web , or you can choose to introduce the packaged JS file directly https://github.com/pili-engineering/QNRTC-Web/blob/master/Release/ Pili-rtc-web.js

Asynchronous processing

Real-time audio and video is a strongly asynchronous scenario, with a variety of operations because the network is all asynchronous related, so that developers can better control the asynchronous logic during code writing. Instead of using the cumbersome callback pattern, the SDK uses modern Javascript's async/await or Promise features to write asynchronous code that avoids the case of a callback to Hell during development (all of the following await codes are assumed to be in an async function package below).

Join the room

When you're ready, the first step is to join the room. Said to be added to the room, the abstract after the statement is actually "what the user in what capacity to join what room." There are 3 unknown quantities: User ID, identity (permission), Room ID. In fact, the whole process of joining the room need a lot of unknown, such as the room to which app (application, different applications in the room independent), the app belongs to which seven cow account and so on. Here we unify these values to encode the signature, become a roomtoken to provide to the front end, only need to pass this token to join the room. (token generation can be done on the seven bull console or dynamically generated using the server-side SDK as needed)

const myRTC = new QNRTC.QNRTCSession(); // 初始化await myRTC.joinRoomWithToken(ROOM_TOKEN); // 加入房间

Capture local media streams

The general collection captures both audio and video, i.e. microphones and cameras, but the SDK also supports either pure audio capture or pure video capture, depending on the requirements. The invocation method is also simple, change the options.

const DOM_ELEMENT = ... // 页面上准备用来播放流的 dom 元素// 采集本地流const localStream = await QNRTC.deviceManager.getLocalStream({    video: { enabled: true },    audio: { enabled: true },});// 播放采集到的流localStream.play(DOM_ELEMENT)

Publishing Media streams

Just get the Stream object as the entry, call the Publish method to

await myRTC.publish(localStream);

Subscribe to Media streaming

When the room is successful, you can always access the users members to get the current room user status, if there is a user in the room is publishing and not himself, then we can initiate a subscription

const users = myRTC.users;users.forEach(async (user) => {    if (user.published && user.userId !== myRTC.userId) {        // 订阅房间其他用户返回的流数据        const remoteStream = await myRTC.subscribe(user.userId);        // 同样，直接调用 play 就可以播放流了        remoteStream.play(DOM_ELEMENT);    } });

Event handling

The SDK exposes a rich list of events to meet the needs of most scenarios, and the handling of events is simple, taking the event "other users publish" as an example

// 监听事件myRTC.on(‘user-publish‘, handleUserPublish);// 只监听一次myRTC.once(‘user-publish‘, handleUserPublish);// 取消某个监听函数myRTC.off(‘user-publish‘, handleUserPublish);// 取消所有监听函数myRTC.removeAllListeners(‘user-publish‘);

The specific list of events can be consulted Https://developer.qiniu.com/rtn/sdk/4423/the-event-list-web

Featured Features

In addition to these basic features, the SDK also provides a number of powerful advanced features to further meet the needs of all walks of life.

Screen sharing

In addition to the direct acquisition of cameras, the SDK also supports screen capture (or window capture) to enable shared screens for meetings. It also enables seamless switching between screen sharing and camera capture, ensuring a user experience.

// 屏幕共享await QNRTC.deviceManager.getLocalStream({    screen: { enabled: true },    audio: { enabled: true },});

Live retweets

For an online meeting, there may be only 10 people involved in the discussion, while most people need to watch the meeting in real time (but not in real-time discussions). This is the intersection of real-time audio and video and live two large scenes, the small number of users with real-time interaction needs to the ultra-low latency (200ms) real-time audio and video cloud, the majority of only real-time viewing needs of users assigned to low latency (2~3s) of the live cloud, can minimize the cost to meet demand. At the same time, real-time video streaming is pushed to the live cloud, and the streaming data can be stored in a storage space for long-term preservation using the API of the seven bull live cloud. Completed a complete business process from real-time interaction to live view to final file storage (for on-demand, etc.).

First of all, in the real-time audio and Video cloud console page associated with the corresponding live cloud space, and then open the push switch.

If you want to push to a custom RTMP address (without using the seven bull Live Cloud), you can also configure it via the back-end API of the real-time AV cloud (see document for details)

After the job is the SDK, the use of the SDK to turn on the live stream is very simple, to join the room after calling a line of code.

myRTC.setDefaultMergeStream(WIDTH, HEIGHT); // 这里的 width height 对应上文设置的合流输出尺寸

With this code, the SDK will default to the average layout of all the streams in the room, and eventually push to the target RTMP address to achieve a full flow push. If you want to customize the screen layout, you can use the following APIs:

myRTC.setMergeStreamLayout("目标用户ID", { w: 100, h: 100, x: 0, y: 0, muted: false, hidden: false });

In addition, we also provide a Web-page real-time whiteboard services, like Zoom, users can share a whiteboard on the page and other users in the room to support demonstrations, while the Whiteboard also supports PowerPoint and PDF presentation. For a demo of this piece, you can visit our Cow class experience https://edu-demo.qnsdk.com.

The above is just a list of conference software commonly used in the functional scene, these basic functions and the user's own scene integration, a simple conference software can be done easily. If you're ready to try and experience it, it's a good choice to visit our Demo (see above). If you are going to connect your products to our live audio and video, here's a more detailed real-time audio and video application development Practice https://developer.qiniu.com/rtn/sdk/5043/ Rtc-application-development-process, from Html/css to JS, has detailed explanations and examples from every line of code to every feature to help you get into it quickly.

From zero to one, develop a Zoom bar with the live audio and video SDK