Analyze audio waveforms, add audio special effects, and add special effects to audio Waveforms

Analyze audio waveforms, add audio special effects, and add special effects to audio Waveforms

I. Preface
Hello everyone, I am a technician of 19944, from Hunan. I am famous for my superb skill. I am currently working in TGideas part-time restructuring. Recently, our Minister aiden was working on a Motion mobile component library, which was not frequently used by many customers.

Previously,
The audio function is relatively simple and requires no precipitation components.
But after some deliberation,
But made a lot of interesting things.

For example:
Analyze audio generation animations;
Some devices play simultaneously (web audio supports simultaneous playback );
Audio features, such as the low voice of a lol hero;
Of course, basic audio playback is the Foundation;
For more details, let the technicians come here one by one.

 

2. If you are interested, ask the guest to continue
Function list:
1. Basic functions-encapsulate basic functions of audio objects
2. Dance with music! -Detect audio Waveforms
3. Make the whole page dance. -Provide music details and event listening (ios8 +, android5.0 + browser <Non-webview>, android2.3 + <chrome, firefox>)
4. Special music effects (ios8 +, android5.0 + browser <Non-webview>, android2.3 + <chrome, firefox>)
5. Support for voice changes (not completed yet)
1. Basic functions1). Function details
Audio Playback, pause, and stop
2) System Support
Android2.3 +, ios6 +
3) Use

var audio = new mo.Audio({src: 'http://ossweb-img.qq.com/images/audio/motion/audio4.mp3',// loop: false,// autoPlay: false});audio.pause();audio.play();audio.stop();

 

2. Dance with music!
1). Function details
Devices that support web audio will analyze audio Frequency Data and generate canvas animations Based on audio Data.
Random waveform animation is used for devices that do not support web audio.
2) System Support
Web audio is not supported:
Android2.3 +, ios6 + (elegant downgrade)
Supports web audio:
Ios8 +, android5.0 + browser <Non-webview>, android2.3 + <chrome, firefox>
3) instance

Http://ossweb-img.qq.com/images/demo/motion/audio/normal.html

4) usage:

window.audio = new mo.Audio({src: 'http://ossweb-img.qq.com/images/audio/motion/audio4.mp3',controller: $('.bg-music')});

5) Core code for function implementation:

self.sound = self.audioContext.createBufferSource();self.audioContext.decodeAudioData(self.arrayBuffer, function(buffer) {sound.buffer = buffer;sound.connect(self.audioContext.destination);self.analyser = audioContext.createAnalyser();self.frequencyData = new Uint8Array(analyser.frequencyBinCount);self.analyser.getByteFrequencyData(self.frequencyData);self.sound.connect(self.analyser);}

 

3. Make the whole page dance. -Provide music details and event listening
1). Function details
The event interface is provided to pass in audio waveform data as parameters to facilitate the control of other elements on the page as the audio.
2) System Support
Ios8 +, android5.0 + browser <Non-webview>, android2.3 + <chrome, firefox>
(The event is not responded to by the system. You can hide the element)
3) instance

Http://ossweb-img.qq.com/images/demo/motion/audio/animate.html

4) Use

Audio. on ('progress', function (e, frequency) {// frequency ratio ('opacity ', frequency [6]/255)(('.ball2').css ('-webkit-transform ', 'translatex ('+ frequency [6]/5 + 'px )');})

5) precautions:
Frequency is an array of 16 characters that contains detailed audio details. If you just do a simple css3 animation that follows the music, you can retrieve the average value of the array.

var total = 0;for(var i = 0; i < frequency.length; i++) {total += frequency[i];};var avg = total/frequency.length;

 

4. Special music effects
1). Function details
This is the most imaginative function, sound conversion. For example, convert a user's voice into a voice similar to a low voice in lol and send it to a circle of friends so that friends can guess who he confessed.
2) System Support
Ios8 +, android5.0 + browser <Non-webview>, android2.3 + <chrome, firefox>
(The system does not support keeping the original audio sound)
3) instance


Http://ossweb-img.qq.com/images/demo/motion/audio/effect.html
4) Use

window.audio = new mo.Audio({src: 'http://ossweb-img.qq.com/images/audio/motion/human-voice.mp4',effect: 'wildecho',controller: $('.bg-music')});

5) technical implementation:
Use the convolver node of web audio. For details, see the source code. Currently, the supported effects include: 'cave ', 'lodge', 'parking', 'lowpass', and 'telphone ', 'spatialized', 'backwards ', 'wildemocho'

 

Iii. Component documentation
Parameters:
Src (required)-the audio address to be played, which must be in the same domain or allow cross-origin requests (accect: */*). If the audio address is a cross-origin audio address, only basic functions are supported;
Controller-container for displaying waveforms;
AutoPlay-automatic playback or not;
Loop-Indicates whether loop playback is enabled;
Effect-effects ('cave ', 'lodge', 'parking', 'lowpass', 'telphone', 'spatialized', 'backwards ', 'wildemocho ');
FillColor-waveform fill color
FillNum-Number of waveforms

Attribute:
CurrentTime-current playback time
Playing-current playback status

Method:
Play-Audio Playback
Pause-audio pause
Stop-audio stop (start from the start point next time)

Event:
Beforeinit-before Initialization
Init-initialization complete
Progress-playing
For example:

Audio. on ('progress', function (e, frequency) {// frequency is a 16-digit combination ('.ball1').css ('opacity ', frequency [6]/255 )})

 

Iv. System Support
Currently, about 35% of devices are fully supported:
Ios8 +, android5.0 + all browsers; chrome and firefox below android5.0.
Devices that are not currently supported will be randomly animated.

 

V. Source Code
Https://github.com/tgideas/motion/blob/master/component/src/main/audio/audio.js

 

6. If you are interested, follow our github address.
Https://github.com/tgideas/motion
This is our only maintenance address. We will continue to add interesting components later.