Summary of developing USB audio device drivers under WinCE

I do not know the USB audio device class before developing the driver for the USB audio device. I thought it was a hid driver and it should be easy to develop, later, I found myself in an unknown field. Thanks to the previous development of the mass storage firmware program and the knowledge of the ov511 USB camera driver, the development was completed even though it took more time and twists and turns in the middle. After the development, I looked back and found that I had a better understanding of the USB protocol and audio processing. This is one of the GAINS. Because I have never done it before, during the development process, I spent 40% of my time thinking about how to proceed and 10% of my time on documenting ideas and coding 50% of the time. I found that the efficiency was good and people were not so tired, this is also the second reward.

1. When summing up development experience, we will first study some basic knowledge points to pave the way.
1. USB protocol
(1) a USB device describes its functions by descriptor. The standard USB device has five types of USB descriptors: device descriptor, configuration descriptor, string descriptor, interface descriptor, and Endpoint descriptor.
(2) a USB device can be regarded as a device that provides multiple serial ports. According to the USB specification, each serial port is called an endpoint to communicate with this endpoint, we need to open the connection to this endpoint, which is the pipeline ).
(3) because a device may have to adapt to multiple situations, multiple endpoints are configured for use. The endpoint settings are called interfaces ). The interfaces shown by the USB device are the interfaces we can find. We can select the interfaces we want to use to find the endpoint, and then open the endpoint to transmit data. Therefore, when the driver starts, you need to record these interfaces.
(4) After opening the endpoint, You can transmit data like a serial port. There are four different USB transmission modes: control transmission, bulk transmission, interrupt transmission, and isochtransfer ). Real-time transmission is required for high real-time requirements such as audio and Viedo.

2. USB driver
(1) hid, USB audio class device, USB video class device, and mass storage devices can be used to develop general device drivers. You only need to modify the vid and PID.
(2) In addition to implementing the stream Driver Interface for a USB device, you also need to implement the three functions: USB installdriver, USB deviceattach, and USB uninstalldriver, And the callback function for pulling out the device.

3. Audio Processing
(1) PCM coding method: pulse coding modulation PCM (Pulse Code Modulation) is actually to digitize the sound for transmission. It is also the most common method. The specific principle can be queried from the Internet.
(2) PCM file: a binary sequence directly formed by Analog-to-analog conversion (A/D conversion). The file does not have an additional file header and end mark.
(3) Fourier Transformation: An Important algorithm in the digital signal processing field. The Fourier principle indicates that any time series or signal of continuous measurement can be expressed as an infinite superposition of sine wave signals of different frequencies. The Fourier Transform algorithm based on this principle calculates the frequency, amplitude, and phase of different sine wave signals in this signal by accumulating the original signals directly measured. The inverse Fourier transformation algorithm corresponds to the Fourier transformation algorithm. In essence, this inverse transformation is also a kind of accumulative processing, so that the separately changed sine wave signal can be converted into a signal. Therefore, it can be said that Fourier transformation converts time-domain signals that were originally difficult to process into frequency-domain signals (signal spectrum) that are easy to analyze. Some tools can be used to process and process these frequency-domain signals. At last, we can use Fourier inverse transformation to convert these frequency-domain signals into time-domain signals.
In mathematics, we can think that any waveform can be expressed by polynomials, that is, mathematical operations can be performed.
(4) The audible acoustic wave ranges from 200Hz to 20 kHz, and the noise in the sound signal is generally in the low frequency section (for example, less than Hz ), after Fourier transformation, this part can be eliminated and then reversed to achieve the goal of de-noise.
(5) Processing audio in Windows is roughly divided into two parts: audio input and output, and ACM compression.

4. Audio driver
(1) The Audio driver processes upper-layer applications through a module of the wave API manager. For detailed structure, refer to msdn
(2) The Audio driver processes PCM data streams. Because the I/O sampling rate of the device is already set in advance, different sampling rates required for upper-layer applications need to be converted. At this time, the linear interpolation algorithm is used to ensure that the audio is not distorted after conversion.
(3) ac97 audio driver structure function analysis.

Ii. device Analysis
Connect the device to the PC and use the tool software bushound and usbview to analyze the information of the device. It has four interfaces (endpoint configuration) as follows:
1. Interface 0: audio control. The endpoint 0 is used for volume adjustment and other operations. Class is 1 (audio) and subclass is 1 (audio control)
Volume Adjustment Process: The default value is interface 0, which sends the set cur command and the volume value. After the volume is changed, the set interface will be sent to interface 2 from interface 0, and then the audio data (PCM wave file) will be sent to interface 2 ), finally, send the set interface from interface 2 to interface 1.
2. Interfaces 1 and 2: audio stream output (speak). Use endpoint 1, class 1 (audio), and subclass 2 (audio streaming ), the former balternatesetting and bnumendpoints are both 0 (zero bandwidth), and the latter are both 1
3. Interfaces 3 and 4: audio input (MIC), endpoint 2, class 1 (audio), subclass 2 (audio streaming ), the former balternatesetting and bnumendpoints are both 0 (zero bandwidth), and the latter are both 1
4. Interface 5: hid. Use endpoint 3. Not used, class is 3 (HID), subclass is 0
5. Data Format type
The input is single-channel, 8 KHz sampling rate, 16-bit PCM, the maximum endpoint transmission value is 16 bytes, the output is dual-channel, 48 KHz sampling rate, 16-bit PCM, and the maximum endpoint transmission value is 192 bytes.
6. Interfaces 1, 2, 3, and 4 have alternative interfaces. They are prepared to start/stop mic or SPK. the USB audio device class specification requires zero bandwidth. Therefore, you only need to perform these operations to enable recording and playback of the device.

Iii. implementation steps and ideas
Step 1: Refer to USB print to implement the driver and provide custom iocontrol code in the driver to supply program calls for device operation. Because it skips the waveapi, It is not universal, this is mainly to test the use of devices, and to deepen understanding of USB device development and accumulate experience.
Step 2: Based on the first step, refer to the wavedev driver. In this way, the driver application only needs to call the functions of the waveapi series to achieve the purpose of operating the device, which is universal.
Step 3: switch between multiple audio devices on the platform.

Iv. Key Issues in Development
Step 1
(1) The sampled PCM file plays faster on the PC.
Cause: the acceleration of playback is caused by data loss. Because the driver's read control code is simply to deliver a real-time transmission request, the application will naturally have a buffer overflow issue when calling it cyclically. Change to use threads to combine events and semaphores in the driver. Up to five real-time transmission requests can be shipped into the asynchronous waiting queue of the kernel, when a dual-end cyclic queue is introduced to manage the buffer zone, there is basically no data loss problem. Note that the value assignment of the dwlen array cannot exceed the maximum transfer of the endpoint, which is also described in msdn.
(2) The sampled PCM file is playing noisy on the PC.
Cause 1: This device sends 960 bytes (10 ms) at a time. Because the package size of the input endpoint is 100, my frame array is 100*10, however, when the data arrives, it is not set to frame [0], but to 96 for each frame. In this way, when the PCM file is generated, 4 more zeros are generated every 96 bytes, after changing the frame array to 96*10, the noise is much lower.
Cause 2: This audio device is powered by 5 v usb, and the 5 V power supply of the Development Board is also used by other devices. There is noise due to hardware reasons. Switching to a product device is basically free of noise.
(3) An exception occurs when Multiple Plugging attempts occur.
In the callback function pulled out by the device, the previously allocated memory, events, semaphores, and open pipelines are not recycled or closed.

Step 2
For the modification of the driver of the wavedev structure, we only need to pay attention to the hardware-related parts, that is, the hwctxt and wavemain parts. For other common parts, we only need to modify the device name in devctxt.
(1) Description of several important functions
Setrate: Set the initial value of the linear interpolation algorithm.
Render2: linear interpolation algorithm for 8-bit, single-channel, and stereo sampling rate conversion.
Transferbuffers: for the input, the data is transferred from the DMA buffer to the application buffer after the sampling rate conversion. For the output, the application buffer is transferred to the DMA buffer after the sampling rate conversion.
(2) modify the wav_init Parameter
The default audio device is loaded when the system starts. According to the wav_init parameter, we can know that the Device Manager is loaded through registerdevice. However, because the USB audio device is dynamically loaded through USB deviceattach, it skips the Device Manager, activatedevice used to load the driver will pass different parameters, so you need to modify the wav_init parameter (or change it to registerdevice, but the former is recommended for msdn)

Note: If you modify the parameter here, the understanding of the wav_init parameter is incorrect. The Device Manager actually loads the parameter through ActivateDeviceEx and passes it to the stream Driver Interface xxx_init, which is actually an active Registry Key Path.
(3) modify the DMA Operation
Whether it is ac97 or IIS audio driver, status switching is performed through a finite state machine. An important mechanism for data transmission is the use of dual-buffering. The function is to transfer data from DMA to CPU. At the first startup, the two buffers are filled up, but only one of them is handed over to the DMA. The following data is converted and moved in the IST to ensure the continuity of transmission, avoid out-of-service DMA during data migration. Because the DMA transmission does not occupy the CPU, the DMA is also in progress when the CPU performs data conversion and migration, so that codec will not pause when there is data playback.
In the USB audio driver, there is no DMA but real-time transmission. If you want to completely copy the working principle of DMA, create a transmission thread to simulate DMA, no matter how large the transmission buffer is, when the transfer callback function activates ist for data conversion and migration, codec has played the data, ist completes data conversion and migration, and notifies the transfer thread to deliver new requests. This process will occupy the CPU, Which is delayed, resulting in a pause. Therefore, the DMA mechanism can only be completely put aside and the implementation mechanism in the first step is adopted, so that there will be no problem. You only need to retain the interface for the start-stop DMA, and change the internal implementation to switch the USB interface to enable the start/stop audio device.

Step 3
You can use waveinmessage and waveoutmessage to switch the audio device. If you confirm that the device is connected, you can directly specify the device ID in waveinopen and waveoutopen. For more information, see the audio device switching code section in the bluetooth module of wince.

5. Problems

1. If you want to change the value of wav_init to the default value during insertion, but do not change the value of wav_deinit when pulling out. It is estimated that the driver has not been loaded yet, but it still needs to be put into the application and dynamically switched through other methods.

2. Switch the default device back to the Message notification function of the USB audio device, and re-insert the device to view the driver loading from the debugging information. wodm_open is also called, but no sound is played. If the items generated under active state are not deleted after Multiple Plugging attempts, the values under the items are deleted. I have read the description in the deactivatedevice source code comment in the private directory: if the item cannot be deleted, the key values under the item are deleted to ensure that the item is accidentally reused, but it does not indicate why the item cannot be deleted. After multiple plug-ins, device.exe and coredll. dll will receive a familiar dataabort error message. Is it because the Device Manager or usbd has not completely released kernel resources related to the device?

Description and solution of this problem:

(1) At first, I thought it was a memory leak problem, and even thought the OHCI driver had a bug. An error occurred while accessing the device linked list of the secondary node. The analysis may be that when the memory is not recycled, the new memory will be inserted and applied for. However, this USB device may not be enough to open memory resources due to several pipelines, I have increased the DMA memory of OHCI In the PDD layer.
(2) Next I found that after the USB audio device is plugged in to load the driver, it will call wav_open, but I don't know who is going to open the device (either waveapi or PM ?), Wav_close was not called during pulling out. I think it may be because the default audio device is always running, so closehandle is not required after createfile. In this case, the handle opened by the USB audio driver through wav_open cannot be released. Is it because of this reason that an exception occurs after Multiple Plugging?
(3) through analysis of waveapi, I finally realized that the previous guess was correct: Why is there wav_open? The first time is PM, it gets the Power Property of the device, and then wav_close. The second time is waveapi, which requires a device handle. Because waveapi communicates with the audio driver through deviceiocontrol, it will not be closed after it is enabled.
I can see from a foreign website that someone has said: note that in general ce hasn't been well tested for hot-unplug of audio devices. The reason may be that the handle is not released after the wavepai enables the audio device.
(4) using the mass storage method, I split the USB audio driver into two DLL files: WAV. DLL and USB. DLL. The former is audio processing implementation, and the latter is loaded when the system starts; the latter is the USB processing part, which provides the USB device operation function interface to WAV. DLL (delayed loading) during device insertion by USB. DLL uses fastcall to pass the context of the USB device to WAV. DLL. the upstream and downstream settings of the USB device in DLL are 0.
In this case, the wav part always exists and interacts with the waveapi. When a USB device is involved in the operation, the system first checks whether the context of the USB device is not empty. Otherwise, the system returns true without processing, the unrelated part of the device is not problematic because it complies with the wavedev structure. Even if the USB device is unplugged, there is no problem when the application switches two audio devices and plays the video. You may need to change the recording.
This may reduce the efficiency, but I cannot think of any other good method at the moment. In addition, I want to add a thread in WAV. DLL to determine USB pulling and insertion. This may enable automatic switching without application switching.

3. The volume of the recording is too small. It is very troublesome to change it to the control panel every time. Consider using APIs or USB commands

Description and solution of this problem:

Because the application requires 8 kHz + 8 bit + mono data, and the device is 16 bits, after the conversion, it becomes the unsigned PCM with a value between 110 and 140, the amplitude is only 30, and the gain is processed as follows: current value = (original value-100) * 4. The amplitude reaches 120 to enlarge the volume. Do not overflow the gain.

4. This device is a USB combination device with audio (USB audio class) and hid functions. The audio function includes one interface for recording and one interface for playing. The hid includes one interface for keyboard, one interface implements the mouse. This part of HID is not identified by the system through debugging, and the phenomenon is usbhid. DLL, kbdhid. the DLL is not loaded. If you put this device in the PC, everything works normally, and the hid of another compound device with the same function works normally. Is it possible that wince does not support combination devices?

6. Reference articles
1. Overview of PCM file formats
Http://blog.csdn.net/c0ffee1982/archive/2007/11/19/1892319.aspx
2. Physical Meaning of Fourier Transformation
Http://blog.csdn.net/wjw2586121/archive/2009/05/02/4142352.aspx
3. Audio Processing in Windows
Http://blog.csdn.net/vavale/articles/306481.aspx
4. wince audio driver
Http://blog.sina.com.cn/s/blog_537bca2a0100ct4h.html ~ Type = v5_one & label = rela_prevarticle
5. ac97 + wm9715 audio driver development
Http:// I .cn.yahoo.com/ROCKFAN_1986/blog/p_35/
6. Linear interpolation algorithm for sample rate conversion in audio drive
Http://topic.csdn.net/u/20090629/14/99352117-12b9-41ce-ba2b-b3d937bacb8e.html
7. Introduction to wince sound-driven model
Http://blog.csdn.net/cnhighway/archive/2009/03/18/4001945.aspx
8. Dual-audio device switching
Http://blog.csdn.net/embest_mhq/archive/2009/04/01/4041860.aspx
9. How to obtain the system audio device list
Http://topic.csdn.net/u/20081006/17/609326BB-A166-4733-A3FE-E68AC9A56811.html
10. wince USB audio/video driver
Http://blog.naver.com/PostView.nhn? Blogid = mkson69 & logno= 30031777873 & widgettypecall = true
11. How to stream isochronous data over Universal Serial Bus
Http://support.microsoft.com/kb/317434/en-us

Original Author link: http://blog.csdn.net/alien75/article/details/4729398