Simplest visual Audio Playback Example 1: general statement

Objective

Recently studied the technology of AV playback under the Windows platform. In the Windows platform video playback technology mainly has the following three kinds: Gdi,direct3d and OpenGL; audio playback technology is mainly DirectSound. These technologies are relatively low-level technologies, so they are more complex to use. I also learned in the process of the lack of a relatively simple and intuitive sample program, so I intend to do some sample programs, while writing some related articles, so that people in this field to learn the relevant knowledge.

I'm going to focus on recording the video playback technology. Previous research has focused on the techniques of video coding and decoding, with little exposure to video playback and display techniques. Previous programs have been using the method called SDL to play the display video screen. In fact, the SDL itself does not have the ability to play the display, it just encapsulates the underlying playback display code. In order to "know the reason why", we need to analyze the video playback of the underlying technology.

The following two techniques are available for video playback in the

under the Windows platform: Direct3D and OpenGL. The above two display technologies are the best. In addition to the above two, there are other options. such as DirectDraw or GDI. But Microsoft has stopped its support for DirectDraw. GDI is not primarily used for video display (mainly for software development). There are more articles about the comparisons between these display techniques, which are no longer in detail here. Plan to write 5 article records three kinds of video display technology: Gdi,direct3d,opengl. The Direct3D contains two simple and complex displays: the use of surface and the use of TEXTURE;OPENGL also contain both simple and complex display methods: Direct portrait and use of texture. Finally, write an article to record the use of SDL2 display video technology, and the above technical code to make a comparison. It is important to note that DIRECT3D,OPENGL these technologies have a large number of functions in the 3D area, but these features are not the focus of these articles. These articles are intended for video technicians rather than 3D game developers, and are therefore more focused on their application in the video display area. For the 3D aspect of the function is basically a stroke.

Let me tell you a little bit about my own feelings. I find it difficult to study direct3d,opengl these display techniques as a video technology person. On the one hand, although there is a large number of DIRECT3D,OPENGL on the network (feeling more than the video codec field of information), but these data are basically for 3D game programming. As a result, you will learn a lot of information that is not relevant to video display, such as projection, light and so on. In spite of this "perseverance" in learning, the final can also master the video display related to the more relevant things (such as the knowledge of texture), but this will consume a lot of time and energy. It is "Time is Money", so learning is not very economical. On the other hand, although it is occasionally possible to find some examples of Opengl,direct3d display video, it is more complicated. These examples are basically the fruits of the work of some "great gods". Not only did they encapsulate many functions for ease of use, but they also took into account a lot of possible mistakes and wrote a lot of wrong-judging statements. This is a good thing for mature applications, but it's a bad thing for beginners. Because too many packages will make it unclear to the novice which functions are DIRECT3D,OPENGL APIs and which functions are added after the "Great God". The code becomes very long after adding too many false judgments and so on, and it's hard for novices to tell which functions are the key APIs. Due to the existence of these problems, so that direct3d,opengl these display technology learning threshold is relatively high, many people are not willing to devote too much energy to study it.

I have written these articles in order to solve the problems. These articles in the knowledge narrative only record Opengl,direct3d and video display related technology, other aspects of the technology as far as possible not to write, so that "kungfu used in the blade"; In the case of code, try to eliminate the insignificant code, leaving only the most critical functions. Believe this will help more people learn video display aspects of the technology.

The sample program in these articles, the input is Yuv/rgb pixel data (note that there is no file header of the kind), the output is a video display window. Because the pixel data volume is larger, the length of the input video is very short, only a few seconds.

Simplest Media Play

The code for playing AV using the Direct3d,opengl,gdi,directsound technology has been integrated into a project called "simplest Media play". The project has been configured and contains the test footage (YUV,RGB,PCM data), which can be run directly to see the results.

SourceForge Project address: https://sourceforge.net/projects/simplestmediaplay/
csdn:http://download.csdn.net/detail/leixiaohua1020/8054395

The above project contains examples of multimedia playback using various APIs (DIRECT3D,OPENGL,GDI,DIRECTSOUND,SDL2). Where the audio input is PCM sampled data. The sound card that is output to the system is played out. The video input is Yuv/rgb pixel data. Output to a window on the monitor to play it out.

Through this project the code beginner can quickly learn to use these APIs to play video and audio techniques.
A total of several sub-projects are included:
Simplest_audio_play_directsound: Use DirectSound to play PCM audio sampling data.
SIMPLEST_AUDIO_PLAY_SDL2: Use SDL2 to play PCM audio sampling data.
Simplest_video_play_direct3d: Use Direct3D's surface to play RGB/YUV video pixel data.
Simplest_video_play_direct3d_texture: Uses Direct3D's texture to play RGB video pixel data.
Simplest_video_play_gdi: Use GDI to play RGB/YUV video pixel data.
Simplest_video_play_opengl: Use OpenGL to play RGB/YUV video pixel data.
Simplest_video_play_opengl_texture: Use OpenGL's texture to play YUV video pixel data.
SIMPLEST_VIDEO_PLAY_SDL2: Use SDL2 to play RGB/YUV video pixel data.

In addition to the source code of several sub-projects, the project includes the following:

Key function call structure diagram

The project contains key function calls for each project. For example, a function call structure diagram that uses Direct3D's surface to play a video. No longer listed.

Test footage

Test_bgr24_320x180.rgb: The order of storage on the hard disk is B1G1R1, B2G2R2, B3g3r3 .....

Test_bgra_320x180.rgb: The order of storage on the hard disk is B1G1R1A1, B2G2R2A2, b3g3r3a3 .....

Test_rgb24_320x180.rgb: The order of storage on the hard disk is R1G1B1, r2g2b2, r3g3b3 .....

TEST_YUV420P_320X180.YUV:

The order of storage on the hard disk is
Yyyyyyyyyy ...
Yyyyyyyyyy ...
Uuuuu ...
VVVVV ...

The contents are as follows:

The resolution is 320x180 and the length is 50 frames (2 seconds). The Forbidden City is cloudy.
Note: You can use Vooya to view RGB/YUV data, which is powerful and convenient. Address: http://www.offminor.de/


NOCTURNENO2INEFLAT_44.1K_S16LE.PCM: The order of storage on the hard disk is L1L1,R1R1,L2L2,R2R2,...

The sample rate is 44.1kHz and the sample bits are 16bit and stereo. The content is the starting fragment in Chopin's Nocturne (Nocturne in E flat major, Op9 No2).

Note: PCM data can be viewed using audio editing software such as audition.

Results

The results of the video display in the project are the same. The pop-up window is the same size as the 500x500, and the content is exactly the same, which also favors the technology between them.

Similarly, the audio playback of the material is identical.

The following shows the results of their operation:

Simplest Audio Play DirectSound

Simplest Audio Play SDL2

Simplest Video Play Direct3D (Surface)

Simplest Video Play Direct3D (Texture)

Simplest Video Play GDI

Simplest Video Play OpenGL

Simplest Video Play OpenGL (Texture)

Simplest Video Play SDL2

from:http://blog.csdn.net/leixiaohua1020/article/details/40246783

Simplest visual Audio Playback Example 1: general statement