Android audio system Notes II: resample-1

By default, the sampling rate of Android sound is fixed to 44.1 kHz, and the recording sampling rate is fixed to 8 kHz, therefore, the underlying audio device drivers only need to set these two fixed sampling rates. If the sampling rate transmitted from the upper layer does not match the sampling rate, the android framework layer performs resample processing on the audio stream.

The resample process is as follows:

 
As the most basic class, audioresample will eventually call this class for playback and recording resample. If you are interested, you can study the resample algorithm and implementation.
Audiomixer is only provided for playback. This type of function is not just a resample, but the sound mixing and volume settings are implemented by this class.

Recording:
1. audioflinger: recordthread is a recording Thread class. When there is a recording request, it enters audioflinger: openinput to create this thread;
2. When this thread is created, call readinputparameters to check whether the upper-layer recording sampling rate is consistent with the fixed recording sampling rate of the underlying audio interface. If not, call audioresampler :: create to create a resampler;
3. For information about audioflinger: recordthread: threadloop, when the recording worker thread starts, the threadloop method will be continuously iterated, mainly as follows: 1) read the underlying audio device to obtain the recording data mbytesread = minput-> Read (mrsmpinbuffer, minputbytes); 2) resample the recording data mresampler-> resample (mrsmpoutbuffer, framesout, this );

Sound releasing:
1. audioflinger: mixerthread is the default playing thread derived from playbackthread, which is created by audioflinger: openoutput;
2. When mixerthread is created, 1) Call readoutputparameters to obtain the fixed sound recording sampling rate of the underlying audio interface; 2) create an audiomixer;
3. About audioflinger: mixerthread: threadloop, when the playing worker thread is started, the treadloop method will be continuously loop, mainly as follows: 1) mix audio data of various tracks (maudiomixer-> process (); 2) write the mixed audio data to the underlying audio device int byteswritten = (INT) moutput-> write (mmixbuffer, mixbuffersize );

Default sampling rate for Android

As mentioned above, the android system uses the audiostreamin: samplerate () and audiostreamout: samplerate () methods to obtain the audio recording sampling rate at 44.1 kHz and 8 kHz by default.

Step by step, we will find that alsa_default.cpp is fixed:[CPP]
View plaincopy

1. // Set the sound releasing Parameters
2. Static alsa_handle_t _ defaultsout = {
3. Module: 0,
4. Devices: audiosystem: device_out_all,
5. Curdev: 0,
6. Curmode: 0,
7. Handle: 0,
8. Format: snd_pcm_format_s16_le, // audiosystem: pcm_16_bit
9. Channels: 2,
10. Samplerate: default_sample_rate, // audio sampling rate, fixed at 44.1 kHz
11. Latency: 200000, // desired delay in USEC
12. Buffersize: default_sample_rate/5, // desired number of samples
13. Modprivate: 0,
14. };
17. // Recording parameter configuration
18. Static alsa_handle_t _ defaultsin = {
19. Module: 0,
20. Devices: audiosystem: device_in_all,
21. Curdev: 0,
22. Curmode: 0,
23. Handle: 0,
24. Format: snd_pcm_format_s16_le, // audiosystem: pcm_16_bit
25. Channels: 1,
26. Samplerate: audiorecord: default_sample_rate, // recording sampling rate, fixed to 8 kHz
27. Latency: 250000, // desired delay in USEC
28. Buffersize: 2048, // desired number of samples
29. Modprivate: 0,
30. };

I have talked about the audio resample method like android on the Internet, which leads to poor sound quality. It is not very difficult to modify it. I will try again later. Idea: First try the upper-layer recording sampling rate to set the underlying audio device. If the setting is successful, no resample is required. If the setting is unsuccessful, the default sampling rate is used. This cannot be changed for the time being, because the sound may need to mix the data of multiple tracks, and the sampling rate of each track is not necessarily the same.

Resample processing in ALSA ??

I use question marks for this chapter, because I am indeed confused here.

Generally, the resample process is as follows:

[Plain]
View plaincopy

1. + --------------------- + --------------------------------- +
2. | App sample rate | Android framework sample rate |
3. | 16 kHz | <-- resample -- | 8 kHz | <-- ALSA Interface
4. + --------------------- + --------------------------------- +

Then ALSA interface can be recorded at 8 KHz sampling rate.

But in fact, I found that the underlying audio device can also use other sampling rates. For example, if the adclrc of I2S is 44.1 kHz, and the codec register is set to 44.1khz, the sound recorded by the Android Application is normal.

The sampling rate set in alsa_default.cpp is 8 kHz, and the setting is successful:

[CPP]
View plaincopy

1. Err = snd_pcm_hw_params_set_rate_near (handle-> handle, hardwareparams,
2. & Requestedrate, 0 );

The returned requestedrate is 8000, which is the fixed recording sampling rate of the android framework.

So where is the resample processing of 44.1 kHz-> 8 kHz? The only one is in ALSA-lib or ALSA-driver, so far I have not found the relevant code.

Bytes --------------------------------------------------------------------------------------------------------------------

2011/10/21

There have been a lot of things recently. Finding a house, renting a house, and moving a house is a mess. It was all done yesterday.

Back to the question, The resample processing for ALSA should be implemented in ALSA-lib. Extract others' analyses as follows:

[Plain]
View plaincopy

1. Snd_pcm_mmap_write_areas () function writes data cyclically until the data is empty. First, find the address mapped to memory PCM-> running_areas, and then call snd_pcm_areas_copy () to convert the data, for example, sample rate and channels (if the source data and hardware support formats are the same, you can simply copy data through memcpy). After converting the data to the data in the hardware support format, call snd_pcm_mmap_commit () write the converted data to the ing memory. The write is completed by snd_pcm_dmix_mmap_commit (). When the data is mixed (do_mix_areas), The ing memory is written.

I cannot guarantee that the above analysis is completely correct (in fact, I tracked the analysis, but there were some discrepancies ). Since ALSA-lib is too bloated and complicated, I am too lazy to analyze it in detail. If I have time to study it, I will make some corrections to this statement.

If we abandon ALSA-lib, we still have other options. Do you still remember the mini ALSA-lib (android2.3.1-Gingerbread/device/Samsung/Crespo/libaudio) mentioned in the Audio System HAL of android2.3? We can analyze from here, because the length may be relatively long, it should be separated into chapters, to be sorted out...

By default, the sampling rate of Android sound is fixed to 44.1 kHz, and the recording sampling rate is fixed to 8 kHz, therefore, the underlying audio device drivers only need to set these two fixed sampling rates. If the sampling rate transmitted from the upper layer does not match the sampling rate, the android framework layer performs resample processing on the audio stream.

The resample process is as follows:

 
As the most basic class, audioresample will eventually call this class for playback and recording resample. If you are interested, you can study the resample algorithm and implementation.
Audiomixer is only provided for playback. This type of function is not just a resample, but the sound mixing and volume settings are implemented by this class.

Recording:
1. audioflinger: recordthread is a recording Thread class. When there is a recording request, it enters audioflinger: openinput to create this thread;
2. When this thread is created, call readinputparameters to check whether the upper-layer recording sampling rate is consistent with the fixed recording sampling rate of the underlying audio interface. If not, call audioresampler :: create to create a resampler;
3. For information about audioflinger: recordthread: threadloop, when the recording worker thread starts, the threadloop method will be continuously iterated, mainly as follows: 1) read the underlying audio device to obtain the recording data mbytesread = minput-> Read (mrsmpinbuffer, minputbytes); 2) resample the recording data mresampler-> resample (mrsmpoutbuffer, framesout, this );

Sound releasing:
1. audioflinger: mixerthread is the default playing thread derived from playbackthread, which is created by audioflinger: openoutput;
2. When mixerthread is created, 1) Call readoutputparameters to obtain the fixed sound recording sampling rate of the underlying audio interface; 2) create an audiomixer;
3. About audioflinger: mixerthread: threadloop, when the playing worker thread is started, the treadloop method will be continuously loop, mainly as follows: 1) mix audio data of various tracks (maudiomixer-> process (); 2) write the mixed audio data to the underlying audio device int byteswritten = (INT) moutput-> write (mmixbuffer, mixbuffersize );

Default sampling rate for Android

As mentioned above, the android system uses the audiostreamin: samplerate () and audiostreamout: samplerate () methods to obtain the audio recording sampling rate at 44.1 kHz and 8 kHz by default.

Step by step, we will find that alsa_default.cpp is fixed:[CPP]
View plaincopy

1. // Set the sound releasing Parameters
2. Static alsa_handle_t _ defaultsout = {
3. Module: 0,
4. Devices: audiosystem: device_out_all,
5. Curdev: 0,
6. Curmode: 0,
7. Handle: 0,
8. Format: snd_pcm_format_s16_le, // audiosystem: pcm_16_bit
9. Channels: 2,
10. Samplerate: default_sample_rate, // audio sampling rate, fixed at 44.1 kHz
11. Latency: 200000, // desired delay in USEC
12. Buffersize: default_sample_rate/5, // desired number of samples
13. Modprivate: 0,
14. };
17. // Recording parameter configuration
18. Static alsa_handle_t _ defaultsin = {
19. Module: 0,
20. Devices: audiosystem: device_in_all,
21. Curdev: 0,
22. Curmode: 0,
23. Handle: 0,
24. Format: snd_pcm_format_s16_le, // audiosystem: pcm_16_bit
25. Channels: 1,
26. Samplerate: audiorecord: default_sample_rate, // recording sampling rate, fixed to 8 kHz
27. Latency: 250000, // desired delay in USEC
28. Buffersize: 2048, // desired number of samples
29. Modprivate: 0,
30. };

I have talked about the audio resample method like android on the Internet, which leads to poor sound quality. It is not very difficult to modify it. I will try again later. Idea: First try the upper-layer recording sampling rate to set the underlying audio device. If the setting is successful, no resample is required. If the setting is unsuccessful, the default sampling rate is used. This cannot be changed for the time being, because the sound may need to mix the data of multiple tracks, and the sampling rate of each track is not necessarily the same.

Resample processing in ALSA ??

I use question marks for this chapter, because I am indeed confused here.

Generally, the resample process is as follows:

[Plain]
View plaincopy

1. + --------------------- + --------------------------------- +
2. | App sample rate | Android framework sample rate |
3. | 16 kHz | <-- resample -- | 8 kHz | <-- ALSA Interface
4. + --------------------- + --------------------------------- +

Then ALSA interface can be recorded at 8 KHz sampling rate.

But in fact, I found that the underlying audio device can also use other sampling rates. For example, if the adclrc of I2S is 44.1 kHz, and the codec register is set to 44.1khz, the sound recorded by the Android Application is normal.

The sampling rate set in alsa_default.cpp is 8 kHz, and the setting is successful:

[CPP]
View plaincopy

1. Err = snd_pcm_hw_params_set_rate_near (handle-> handle, hardwareparams,
2. & Requestedrate, 0 );

The returned requestedrate is 8000, which is the fixed recording sampling rate of the android framework.

So where is the resample processing of 44.1 kHz-> 8 kHz? The only one is in ALSA-lib or ALSA-driver, so far I have not found the relevant code.

Bytes --------------------------------------------------------------------------------------------------------------------

2011/10/21

There have been a lot of things recently. Finding a house, renting a house, and moving a house is a mess. It was all done yesterday.

Back to the question, The resample processing for ALSA should be implemented in ALSA-lib. Extract others' analyses as follows:

[Plain]
View plaincopy

1. Snd_pcm_mmap_write_areas () function writes data cyclically until the data is empty. First, find the address mapped to memory PCM-> running_areas, and then call snd_pcm_areas_copy () to convert the data, for example, sample rate and channels (if the source data and hardware support formats are the same, you can simply copy data through memcpy). After converting the data to the data in the hardware support format, call snd_pcm_mmap_commit () write the converted data to the ing memory. The write is completed by snd_pcm_dmix_mmap_commit (). When the data is mixed (do_mix_areas), The ing memory is written.

I cannot guarantee that the above analysis is completely correct (in fact, I tracked the analysis, but there were some discrepancies ). Since ALSA-lib is too bloated and complicated, I am too lazy to analyze it in detail. If I have time to study it, I will make some corrections to this statement.

If we abandon ALSA-lib, we still have other options. Do you still remember the mini ALSA-lib (android2.3.1-Gingerbread/device/Samsung/Crespo/libaudio) mentioned in the Audio System HAL of android2.3? We can analyze from here, because the length may be relatively long, it should be separated into chapters, to be sorted out...