Capturing audio and video information using HTML 5

For a long time, the capture of audio and video information has been a difficult point in web development. For many years, we have relied on browser plugins to implement this requirement.

In HTML 5, there are many APIs that can access hardware devices, such as the Geolocation API to access the GPS device, the orientation API to access the accelerometer device, the WebGL API to access the GPU device, Access the audio playback device's web Voice API and more. These APIs are very powerful, because developers can directly write Javasccript script code to access the underlying hardware devices.

This article describes a new API that uses the Navigatior.getusermedia () method to give Web applications the ability to access the user's webcam and microphone devices.

Technology development History of capturing media data

In the past few years, there has been a need to access client-side devices in Web applications, so the organization decided to organize a DAP working group to develop a unified standard for the implementation of this requirement.

Next, let's look at what happened in 2011:

Capturing media data in an HTML paging file

The first standard to be developed by the DAP Working Group is how to capture media data in the HTML page of a Web application. They decided to overload the input element of type file (<input type= "File" >) and add a new property value for the Accept property.

If the developer wants to implement the camera's ability to take pictures, write the code shown below.

<input type= "File" accept= "Image/*;capture=camera" >

The code for recording video data and audio data is similar to the following:

<input type= "File" accept= "Video/*;capture=camcorder" ><input type= "file" accept= "audio/*;capture= Microphone ">

In this code, you can simply use the file control (INPUT element of type file) to complete the ability to take pictures or record media data. But the ability to implement related requirements, such as rendering captured video data in a canvas element, or applying a WEBGL filter to captured video data, is not widely used by developers because of the lack of such code.

Browser support:

• Android 3.0 Browser
• Chrome for Android (0.16)
• Firefox Mobile 10.0

Device element

If you use the file control, the ability to handle it after capturing media data is very limited, so there is a new standard that can support any device. The standard uses device elements.

Opera browser is the first browser to achieve video data capture via device element. Almost on the same day, the WHATWG organization decided to use the Navigator.getusermedia () method to capture media data. One weeks later, Opera launched a new browser that supports the Navigator.getusermedia () method. Later, Microsoft tools introduced the IE 9 browser that supported the method.

The device element is used as follows.

<device type= "Media" onchange= "Update (this.data)" ></device><video autoplay></video>< Script>function Update (stream) {    document.queryselector (' video '). src = stream.url;} </script>

Browser support:

Unfortunately, there is no official version of the browser that supports device elements yet.

WEBRTC

Recently, due to the advent of WEBRTC (Web real time Communication:web Live communication) API, media data capture technology has a great development. Companies such as Google, Opera, and Mozilla are trying to implement them in their own browsers.

The WebRTC API is an API that is closely related to the getusermedia approach, providing the ability to access the client's local camera or microphone device.

Browser support:

So far, in the Chrome version 18 browser, the Chrome://flags page is set up to use WEBRTC, which is used by default in the Chrome 21 version of the browser and is no longer required for setup. The WEBRTC API is supported by default in Opera 12 and Firefox version 17 browsers.

Using the Getusermedia method

By using the Getusermedia method, we can directly access the client's local camera device and microphone device without relying on plugins.

Detect browser support

You can detect whether the browser supports the Getusermedia method by using the method shown below.

function Hasgetusermedia () {    //Please note: Do not use the prefix in Opera browser    return!! (Navigator.getusermedia | | navigator.webkitgetusermedia | |    Navigator.mozgetusermedia | | Navigator.msgetusermedia);} if (Hasgetusermedia ()) {    alert (' Your browser supports Getusermedia method ');} else {    alert (' Your browser does not support Getusermedia method ');}

Get permission to access a device

In order to access the client camera device and the microphone device, we first need to get permission. The first parameter of the Getusermedia method is an object that specifies the media type. For example, when you want to access a camera device, the first parameter should be {video:true}, in order to access both the camera device and the microphone device, you need to use the {video:true,audio:true} parameter, as shown in the code:

<video AutoPlay id= "video" ></video><script>var Onfailsohard = function () {    alert (' Device denied access ');};/ /Do not use vendor prefix Navigator.getusermedia ({video:true, audio:true}, function (Localmediastream) {    var video = document.getElementById (' video ');    video.src = window. Url.createobjecturl (localmediastream);    Note: When using the Getusermedia method, the Onloadedmetadata event is not triggered in the Chrome browser    video.onloadedmetadata = function (e) {        ///subsequent code slightly    };}, Onfailsohard);</script>

In this code, the use of the video element is combined. Note that we did not use the SRC attribute value of the video element, but instead specified a URL address for the video element that referenced the media file, and converted the Localmediastream object that represents the data obtained from the camera to a BLOB URL.

In this code, the AutoPlay property is used for the video element at the same time, and if the property is not used, the video element stays at the first frame captured.

Please note: In Chrome, if you use only {audio:true}, the audio element is not available in Opera browser.

If you want multiple browsers to support the Getusermedia method at the same time, use the code shown below:

Window. URL = window. URL | | Window.webkiturl;navigator.getusermedia  = Navigator.getusermedia | | navigator.webkitgetusermedia | |                          Navigator.mozgetusermedia | | Navigator.msgetusermedia;var video = document.getElementById (' video '); if (Navigator.getusermedia) {    Navigator.getusermedia ({audio:true, video:true}, function (stream) {        video.src = window. Url.createobjecturl (stream);    }, Onfailsohard);} else {    alert (' Your browser does not support the Getusermedia method ');}

Security

In some browsers, when the Getusermedia method is called, a prompt window is displayed asking the user whether to allow or deny access to their webcam or microphone. As shown in.

Photo

In the canvas API, you can use the Ctx.drawimage (video,0,0) method to output a frame screen from a video element to a canvas element. Of course, now that we've exported the image information from the captured user's camera to the video element, we can also export the image information to the canvas element through the video element, which is the real-time camera function, as shown in the code below.

<video autoplay></video></img><canvas style= "display:none;" id= " Canvas "></canvas>var video = document.getElementById (' video '); var canvas = document.getElementById (' Canvas ' var ctx = Canvas.getcontext (' 2d '), var localmediastream = Null;function Snapshot () {    if (localmediastream) {        Ctx.drawimage (video, 0, 0);        document.getElementById (' img '). src = Canvas.todataurl (' Image/png ');}    } Video.addeventlistener (' Click ', snapshot, false);//Do not use the vendor prefix Navigator.getusermedia ({video:true}, function (stream) {    video.src = window. Url.createobjecturl (stream);    Localmediastream = stream;}, Onfailsohard);

Apply CSS Filters

CSS filters can now be used in Chrome version 18 and above.

With the use of CSS filters, we can add a variety of image filter effects to videos captured in the video element.

<style> #video3 {    width:307px;    height:250px;    Background:rgba (255,255,255,0.5);    border:1px solid #ccc;}. Grayscale {    -webkit-filter:grayscale (1);}. Sepia {    -webkit-filter:sepia (1);}. Blur {    -webkit-filter:blur (3px);} ... </style><video id= "video" Autoplay></video><script>var idx = 0;var filters = [' Grayscale ', ' Sepia ', ' blur ', ' brightness ', ' contrast ', ' hue-rotate ',               ' hue-rotate2 ', ' hue-rotate3 ', ' saturate ', ' invert ', ']; function Changefilter (e) {    var el = e.target;    El.classname = ";    var effect = filters[idx++% filters.length]; Loop through filters.    if (effect) {        el.classList.add (effect);    }} document.getElementById (' video '). AddEventListener (' Click ', Changefilter, False);</script>

Capturing audio and video information using HTML 5