I.
Reasons for writing a playerIn the diagnosis of video conferencing media problems, it is often necessary to take the IP network traffic and form the original code stream for the player to play and diagnose the problem based on the playback effect. This process is generally divided into the following steps: 1) First, use Ethereal to determine whether the RTP stream matches the signaling, such as media type, media Encapsulation Format, and media transport layer address. This step mainly diagnoses the problem of signaling media cooperation and intercommunication. 2) use Ethereal's RTP analysis tool to determine whether there is RTP packet loss or serious jitter, because these will also affect the effect of the video conference. This part mainly diagnoses network problems. 3) extract the pure bitstream using a bitstream Extraction Tool compiled based on Winpcap and send it to the PC decoder after proper processing (the image can be C03 of the image group or open-source ffplay, etc, audio can be played using audacity) to watch the playback effect. Whether or not there are major I frames and image quality problems. This feature is often used to diagnose the coordination between the decoder and the master. This step involves a lot of issues, and the following lists the issues that need attention: a) for H.261, H.263 and other code streams, You need to note the splicing of sbit and ebit, for H.263 + code streams, simply follow the specifications, for H. 264 code stream. Add 0 x, 0 x, 0x00, and 0x00 before each nal unit to help decoder decoding. Note that H. 264 The bitstream must have an image parameter set PPs and a sequence parameter set SPS before it can be correctly decoded. If there is no need for a parameter set, you can try to configure a default parameter set. B) for audio, you can use some auxiliary libraries to convert g.722, g.728, g.723.1 and g.729 into signed 16-bit PCM and then play back the audio. Note that the sampling rate of g.722 is 14 K, it sounds strange to play 8 K. When using audacity, You can first listen to the abnormal sound segment, and then look at the corresponding waveform to see if there is a packet loss problem, finally, find the corresponding package in the capture package file according to the time displayed by audacity to check whether there is a problem of high jitter or packet loss. From the content listed above, we can see that it takes some steps and techniques to extract and play code streams, and involves some tools. When using FFMPEG or mplayer, some parameters need to be remembered. In the import dialog box of audacity, some bitstream knowledge is also required, which is difficult for technicians who are not very familiar with the media. For this purpose, an integrated tool is compiled. The tool operates on the network packet capture object, and then automatically aggregates the relevant code streams based on the transport address. You only need to select the corresponding code streams, this code stream can be played. If the QoS algorithm is embedded, the actual bitstream can be simulated to cause packet loss. It is suitable for general engineers to locate problems.
II.
Prepare the player1) Use wxWidgets for GUI library display 2) use FFMPEG for decoding library 3) Use SDL for graphic display library 4) the packet capture library uses Winpcap as the library to manipulate the packet capture library 5) The use of the audio decoding library uses a commercial trial library, imtelephone's easy series library 6) the audio playing library uses the portaudio library to play the audio.
III.
Player OverviewBasic Steps for player operations: 1) first use the Winpcap library to read the packet capture file and classify the bitstream into the memory according to the transfer address. 2) based on certain rules, we can infer the type of the bitstream contained by each transport address. If it is a fixed load type, the type is provided directly. If it is a dynamic load type, it is inferred based on prior knowledge (currently, the video's h263 + and h264, ). 3) after you select QoS simulation, perform step-by-step simulation based on the time clues of the packets stored in the memory and the QoS scheduling time to properly record each packet loss, after the simulation is completed, the summary information is output. 4) if the user chooses to play the video, if the user has already opened the packet capture file and selected it, ask the user whether to play the selected video or open the file on the disk. Although there are some differences, there is no big difference for players. The following two branches are described as follows: a) if the stream playback file is used, note that preprocessing is required before sending it to the decoder. For videos, for example, the strip of H.261 and H.263 load headers and the splicing of sbit and ebit, the simple change of H.263 + head, H. add the stream header of 264. For audio, when the audio library needs data, the data needs to be decoded and then provided to the audio for playing in the callback. B) if you are playing a disk file, you need to prompt the user to select the file in some areas that cannot be confirmed, such as 5.3k and 6.3 K of g.723. After the user confirms the file, it will be well handled. Key points in player programming: 1) Use of FFMPEG library, such as library initialization, how to start a specific decoder, how to obtain decoded frames, and how to process function return values, and how the data structure contains the image data. Only by understanding this can the image be solved smoothly. 2) how to draw an image in wxpanel is mainly to use sdl_surface to fill in the decoded data into sdl_surface, and then generate a buffered wxbitmap using sdl_surface data during re-painting, then draw the image directly. 3) when playing audio, you must correctly understand the sampling rate, single-and dual-channel, and sampling data type. In the callback function, you must be able to correctly assign values to the buffer. 4) in the wxWidgets environment, the correct thread is used. Because image playback requires timed driving, a thread must be derived to generate scheduled events. At the same time, you must be cautious when processing synchronization between threads. Otherwise, deadlock or other exceptions may occur.
