Myvoix2.0.js Source Analysis Webspeech and Webaudio

Wedge

With the advent of the mobile internet era, various mobile devices have come into our lives. Whether it's a mobile phone in everyday life, a variety of smart wristbands for night runners, or a futuristic Google Glass, they are changing our habits and habits of user interaction. The touchscreen replaces the physical button, and Siri starts to release our hands slowly, and the hardware like leap Motion lets us completely eliminate the need to touch it devices to control them with gestures. In such a background, the front-end interaction will involve more and more meta-interdisciplinary, we like more than 10 years ago people experience the birth of CSS, witnessed a drive the entire industry and even the transformation of society.

Drop IE6 as you drop the table layout

If today your work is still only rigorous daily to compatible with IE6, then it is time to get up and look at the scenery around, HTML5 has been far more than just those gradients and rounded corners. In this blog post, I would like to introduce two sets of APIs, Web speech and web audio, which are a core part of the Myvoix.js framework, which has been implemented in several earlier versions of Chrome, and you can call them by webkit prefixes.

Web Speech

Webspeech API is published by Speech API Community Group, the main function is to convert speech input into text, if you can add a semantic analysis of the server, then it is a Siri. You can also simply interpret it as a programmable input voice input box (the following code).

<typex-webkit-speechlang/>

In the era of Siri just born, the above line of code is the front-end interface to make the cock silk high-handsome weapon, the boss did not know you just wrote a line of code to make a voice input box (secretly on the input to add X-webkit-speech students please praise). However, only one input box label is obviously unable to meet the programmer's burning control of the code, so the Web speech came into being.

Back to the code, to use the Webspeech API we first create a Window.webkitspeechrecognition object.

1 var window.webkitspeechrecognition (); 2     _rec.continuous=true; 3     _rec.interimresults=false; 4     _rec.lang= ' en-us '; 5     _rec.maxalternatives=1;

For everyone to see clearly, here I slightly modified the source code in the Myvoix. After you see the new SpeechRecognition instance, there are several parameters that we need to configure.

Continuous: If set to false, then when the instance is started, it will return immediately if no input or error is entered. This needs to wait until the meaningful input is returned, so we set true.

Interimresults: If set to true, then the words in the parsing process will be returned when the Onresult is in progress. This only requires the final analysis result, so set to false.

Lang: This is language, everyone should read it.

Maxalternatives: Sets the maximum speechrecognitionalternatives for each result.

Next we invoke the instance's Start method to turn on the Speechdrecognition listener. However, you will need to specify the instance's Onresult event before, and the word after the completion of speech parsing is passed into this method.

1_rec.onresult=function(Eve) {2     varLen =Eve.results.length,3i =Eve.resultindex,4j = 0,5 listeners,6 command;7      for(i; i < len; i + = 1) {8         if(eve.results[i].isfinal) {9         //Get WordsTenCommand = Eve.results[i][0].transcript.replace (/^\s+|\s+$/g, "). toLowerCase (); One         if(console.log) { AConsole.log (eve.results[i][0].transcript);  -         }    -         //your code here .... the     } -     } - }; -_rec.start ();

There is a schema for the binding of the word event in Myvoix, and then the post has the opportunity to detail it.

Web Audio

With the speech recognition in order, we need to get the input signal of the microphone, so that we can achieve the Myvoix waveform function. If you've ever called a webcam with JavaScript, you must have used Navigator.webkitgetusermedia, which is what you need to get the microphone's sound source data in Web audio. First look at the source code in Myvoix:

1Navigator.webkitgetusermedia ({audio:true},function(e) {2               varContext =NewWebkitaudiocontext (),3Javascriptnode = Context.createscriptprocessor (2048, 1, 1),4Audioinput =Context.createmediastreamsource (e),5Analyser =Context.createanalyser (),6Splitter =Context.createchannelsplitter ();7Analyser.smoothingtimeconstant = 0.3;8Analyser.fftsize = 1024; 9 Audioinput.connect (splitter);TenSplitter.connect (analyser,0,0); One Analyser.connect (javascriptnode); A Javascriptnode.connect (context.destination); -                              -Javascriptnode.onaudioprocess =function(e) { the                       varArray =NewUint8array (analyser.frequencybincount); - Analyser.getbytefrequencydata (array); -                       varAverage = Me.getaveragevolume (e.inputbuffer.getchanneldata (0)); -                      if(Average > 0) { + me.changenoise (average); - me.changefrequence (average); +                      } A     } at},function(){});

At first glance, a bit like WebGL, you need to link a bunch of stuff. Further analysis of the code:

Navigator.webkitgetusermedia ({audio:true},function(e) {    //Success Callback    //},function() {    //Error callback     //};

The first step uses the Webkitgetusermedia object to invoke the local microphone, and the main code is implemented in the success callback function.

var New Webkitaudiocontext (),     = Context.createmediastreamsource (e);

Then we need to create a Webkitaudiocontext instance where many useful components can be created. Here you can create an input source by using the Createmediastreamsource method and the parameters in the Getusermedia success callback function. Passing through a layer of layer can eventually connect to the output position of the context.destination. Let's simply go through a few of the nodes used in Myvoix:

Analyser: This one look is the analysis of the source used, the specific general use in the sound visualization.

Splitter: This node is used for channel conversion, in Myvoix we use it to turn the source into the left and right two channels.

Javascriptnode: This node is used for JavaScript-level listener processing. With the Onaudioprocess function, we call the function that draws the waveform each time the sound is sampled, and eventually it connects to the output.

In the MyVoix2.0, roughly only the nodes created by the above Audiocontext are used. By adding other nodes in a similar way, the Web audio can also be used to locate and play the sound in 3D space.

　

End

This article introduces the two core technologies used in Myvoix, and through this blog post I hope you have a general understanding of the implementation of the speech technology in the HTML side. We write in the garden, do not have to fight like Lu Xun, but also look forward to the technology to advance this era.

Myvoix Source Address

　　forwarding Please specify the source :http://www.cnblogs.com/Arthus/p/3889457.html