[Reprint] reasons for intermittent audio on the player (RealMedia)

It took me three weeks to solve this problem. When the player of the current product solves the real file with a high resolution and a high bit rate, half of the audio playback will be disconnected, and a small buffer data will be lost, and then continue. The breaking frequency is proportional to the bit rate.

Since there is a problem with the audio, of course we should first start to check the audio, so from Demux, to decode, and then to PCM render, I flipped it over, the input and output of each filter are filled with debugging information and timestamp printing. There is still no result after tracing for two weeks. There is no problem with audio. However, it can be confirmed that the audio decoding is too late. The previous buffer duration has been completed, and the data in the next buffer has not arrived, this will cause this situation.

Finally, the solution provider confirmed that there are only two possibilities: one is that the input/output buffer of fitler is empty/full, leading to CPU blocking not working, and the other is that since the CPU is fully loaded, this is the performance issue. Currently, the CPU clock speed cannot withstand high bit rate decoding.

1. Solve the problem of preventing CPU block first. When the CPU time slice is tight, the buffer pool is full and is sensitive to the priority setting of the filter! Demux must be of the highest priority; otherwise, you will not be able to play it later. Let's look at the following filters connection diagram. It should be said that it is the most prototype filter of the player.

(1) If both video decoder and audio decoder are of medium priority and render is of high priority, Video Decoder does not have enough time slice for decoding, as a result, the video bitstream buffer pool is quickly filled up by Demux, which makes Demux unable to plug in the later buffer, so Demux is stuck there waiting for the video bitstream buffer pool to be dissatisfied, at this time, the file reading has been blocked, and the next piece of data is probably audio, but the file cannot be read due to file offset. The audio decoding on the other is faster, therefore, the audio bitstream buffer pool is empty, and audio decoder has no data to solve, blocks, and audio breaks.

(2) If both video decoder and audio decoder have a high priority and render is a medium priority, Demux will connect the two decoder to run well and decode quickly, however, the render does not get enough time slice to send the decoded PCM data buffer, so the audio is disconnected. At the same time, because the audio PCM buffer pool is full, a lot of PCM data cannot be output, so when audio decoder sends the buffer, It is stuck waiting for the notification of non-empty events in the render input pool, therefore, audio decoder is blocked, audio bitstream buffer pool is full, and Demux is stuck again. Haha, it's all done

(3) There are other matching methods, namely, the Priority Ratio between video decoder, audio decoder, and PCM render. I feel like a high school physical experiment is adjusting the three-legged level, how to adjust it cannot make the small bubbles of the level in the middle. After three liters of vomiting blood, I finally chose video decoder and audio decoder to both be of medium priority and render to be of high priority. This ratio was also recommended by the solution provider. At that time, I asked the other party if there were any formulas for such adjustments. He said that there was no formula, and there was only experience. I tried to adjust it based on experience. I was crazy about faint. If someone asks me if I have any formula, I will also tell him: experience depends on experience!

2. After the above adjustment, the audio is still disconnected, so you must begin to improve performance. The simplest way is to speed up the CPU, but it cannot be done now. Another better way is to discard some video frames before the decoder. I just tried to lose half of the frames before decoder, AND THE EFFECT looks pretty good. Although theoretically the frame rate drops to 12fps, there seems to be no difference to the naked eye. The most important thing is that, the audio keeps playing, Which is perfect.

Therefore, in solving this problem, I actually took a lot of detours. From the very beginning, I didn't realize this was a system-level problem. I thought that the audio module should be checked to adjust the buffer pool size and the priority of each filter, and finally add frame drop processing, the problem can be solved. Finally, it seems that the audio disconnection problem is not necessarily caused by audio decoding itself. It depends on the face of others, whether Demux is stuck, and whether Decoder gets enough input data; whether the render is stuck, and whether the decode data is processed in time to free up the buffer. Even if audio decoder has a high priority, but the front and back filters are stuck, the audio will also be disconnected. The entire audio stream is affected and restricted by the video decoder chain. This reminds me of the old saying: "How much water can be loaded in a bucket depends on the shortest piece of wood." It is no longer appropriate to describe this.

 

Link: http://www.cnblogs.com/walzer/articles/733903.html