Reprint: VLC Architecture and process analysis

This is a reprint of the article, but did not find the source, so if the author saw this article, please contact me, I will follow your wishes or delete, or attach your original link! 0x00 Front-facing information

VLC is a very large project, I start with its architecture and process analysis, involving a few very fine concepts on the aside, later detailed analysis.

0X01 source Structure (Android Java-related temporary analysis)

# build-android-arm-linux-androideabi/: Third-party libraries. # modules/: module code. # Modules/demux: The code for the multiplexed module. # Modules/codec: Decode module related code. # modules/access: Access module-related code. # Other: not detailed analysis. # src/: VLC architecture core code. # src/config/: Load configuration from command line and configuration file, provide read and write configuration of function module. # src/control/: Provides motion control functions such as play/pause, volume management, fullscreen, log, etc. # src/extras/: platform-specific related code. # src/modules/: module management. # src/network/: Provides a network interface. # src/posix/: Multithreading-related. # src/osd/: Displays the action on the screen. # src/interface/: Provides the code in the interface that can be called, such as the key after the hardware responds. # src/playlist/: Manage playback functions such as stop, play, next song, Shuffle, etc. # src/text/: Character Set. # src/input/: Enter the stream-related code. # src/video_output/: Initialize the video player to play after data processing from the decoder. # src/audio_output/: Initializes the audio mixer and plays back the data obtained from the decoder. # src/stream_output/: Output audio stream and video streaming to the network. # src/test/: LIBVLC test Module. # src/misc/: LIBVLC uses other parts of the functionality, such as threading system, Message Queuing, CPU detection, object lookup system, or platform specific code. # Other: not detailed analysis.

0X02 Basic Concepts

For the playback of a video, the player performs the following steps roughly as follows:

1. Read RAW data
2. Solution multiplexing
3. Decoding
4. Show

Based on the above concepts, VLC abstracts several other concepts, first listing the important concepts that are abstracted from VLC:

1. Playlist:playlist represents a playlist, and VLC creates a playlist thread after startup, creating input dynamically after the user enters it. The
2. input:input represents input, and when the user enters a file or stream address through the interface, input thread is created dynamically, and the thread's life cycle is until the end of the session. The
3. access:access represents access, a layer of VLC abstraction that directly uses the file or network IO interface down, providing an IO interface to the stream layer. The
4. Stream:stream represents a stream, a layer of VLC abstraction that directly uses the IO interface provided by the access layer and provides an IO interface up to the DEMUX layer service. The
5. Demux:demux, which is a concept in video technology, uses the IO interface provided by the stream layer down directly, and the data is sent Es_out. The
6. es_out:es_out represents the output, which is a layer of VLC abstraction that acquires the DEMUX data and sends decode decoding. The
7. Decode:decode is a decoding, a concept in video technology that acquires data from Es_out (via a FIFO interaction) and decodes and sends output. The
8. output:output represents the output, obtains the data from the decode, and sends the readerer. The
9. Readerer:readerer represents the display, obtains data from output (through a FIFO interaction), and then displays.

Shows the relationship of these abstract concepts, where blue represents the concept of VLC abstraction.

A review of the 0x04 architecture

VLC's overall framework is designed as a set of module management mechanisms that classify functions and abstract them into modules.

The main of VLC Main:player. Initializes the LIBVLC and loads the user interface.
The core of LIBVLCCORE:LIBVLC, which abstracts out a libvlc_instance_t object, provides a modules loading/unloading mechanism.
Modules:modules provides specific functionality, such as the above Access,demux,decode is in the form of a module exists.
External Libraries: External open Source Library.

How the module is loaded:
First the module registers itself in VLC, code snippets such as:

vlc_module_begin()...vlc_module_end()

Then, when the module needs to be loaded, call the Module_need interface to find the appropriate module. Once the appropriate module is found, the callback method set in the registration is executed, such as the open* name method.
You can also implement the module yourself, just follow the VLC module standard. Many of the modules in VLC are implemented through external open source libraries.

The modules in VLC are broadly categorized:

0X05 Process Analysis

First, the flow chart, referring to the diagram, and then continue the following process analysis, Green Line to open VLC after the execution of the operation; The black line indicates that the user entered a video after the operation; the blue Line begins with the red circle, which indicates the flow of data after the input stream has started playing.

(1) Main function (VLC/BIN/VLC.C)

1. Parameter signal processing related, unknown analysis.
2. Call Libvlc_new () to initialize a libvlc_instance_t instance. (libvlc_instance_t is opaque. It represents a LIBVLC instance)
   2.1 Call Libvlc_internalcreate to create a libvlc_int_t. (This structure was a LIBVLC instance, for use by LIBVLC Core and plugins.)
   2.2 Call Libvlc_internalinit to initialize the libvlc_int_t instance.
   2.3 Initialize the other members of the libvlc_instance_t.
3. Call Libvlc_set_exit_handler to set the callback function when VLC exits.
4. Call libvlc_add_intf to add a module.
   4.1 Gets playlist, if empty, calls Playlist_create to create a playlist structure and calls Playlist_activate to create a new playlist thread (src/playlist/ THREAD.C).
   4.2 Call Intf_create to create a default interface.
   4.2.1 calls Vlc_custom_create to create a VLC object (intf_thread_t).
   4.2.2 Registers a callback method that adds a interface.
   4.2.3 calls Module_need to load a interface module.
5. Call Libvlc_playlist_play, play playlist content if the playlist is not empty and is set to AutoPlay.
6. Signal processing related, unknown analysis.

(2) Create an input

1. After the initialization is successful, the program runs in thread threads (SRC/PLAYLIST/THREAD.C) of playlist, looping through the requests entered by the interface.
2. When entering a new file or stream address, the Playlistvacontrol obtains the signal and sends the signal.
3. After the thread receives the playback request, the Playitem method is called in Looprequest.
   3.1 Call Input_create to create an input structure and initialize various members, including calling Input_esoutnew to create P_es_out_display (es_out).
   3.2 Call Input_start to create an input thread run (src/input/input.c).

(3) Initialize input

Call the Init method in Run (SRC/INPUT/INPUT.C) to begin the initialization process.

2. Call Input_esouttimeshiftnew to create a new 50M timeshift (pause cache), including the creation and initialization of p_es_out (Es_out), which is related to step 9.
4. Set the status of input to opening_s.
5. Call Inputsourceinit.
   3.1 Call INPUT_SPLITMRL to explode the input URI.
   3.2 Call Demux_new to load the "Access_demux" module with the stream parameter null.
   3.3 If there is no appropriate "Access_demux" module, call Access_new to create an actual access.
   3.3.1 calls Vlc_custom_create to create the access_t struct.
   3.3.2 calls Module_need to load the appropriate access module.
   3.3.3 calls the open* method of the Access module, taking the Avio module as an example.
   3.3.3.1 calls Vlc_init_avformat to initialize VLC as the Avformat environment.
   3.3.3.2 calls Avio_open2 to open the URI.
   3.3.3.3 sets the IO method pointer for access.
   3.4 Call Stream_accessnew to create a stream.
   3.4.1 Sets the IO method pointer for the steam layer based on the mode (Stream/block). The Io method of the stream layer actually points to the IO method pointer corresponding to the access layer.
   3.4.2 the buffer request for the stream layer and initializes the memory.
   3.4.3 calls Astreamprebufferstream to perform a read operation.
   3.5 Call Stream_filterchainnew,add Stream Filters (source description).
   3.6 Call Demux_new to create a demux.
   3.6.1 calls Vlc_custom_create to create the demux_t struct.
   3.6.2 calls Module_need to load the appropriate Demux module.
   3.6.3 calls the open* method of the Demux module, taking the Avformat/demux module as an example.
   3.6.3.1 calls Stream_peek to fetch data from the stream layer for parsing the input file format.
   3.6.3.2 invokes Av_probe_input_format parsing the input file format.
   3.6.3.3 sets the value of the demux_sys_t struct part variable.
   3.6.3.4 calls Avformat_alloc_context to assign the Avformatcontext struct body.
   3.6.3.5 calls Avio_alloc_context to set the AVFORMATCONTEXT structure of the Aviocontext type member PB, and sets the read and Seek method pointers.
   3.6.3.6 call Avformat_open_input to open an input, here input and VLC in the input is not a concept, about avformat_open_input analysis See my other article "Avformat_open_ Input detailed analysis of the link address.
   3.6.3.7 invokes the Avformat_find_stream_info parsing stream information, which reads the data initialization stream and the stream decodes the information.
   3.6.3.8 sets the FMT variable based on the stream information analyzed, and calls Es_out_add.
   3.6.3.9 actually calls Esoutadd (SRC/INPUT/ES_OUT.C), adds a es_out, has several streams to do several es_out_add operations, such as a video stream and an audio stream in the input, then two es_out_add operations.
   3.6.3.10 Nb_chapters related not detailed analysis.
   3.7 Set record correlation.
   3.8 Call Demux_control to set Demux pts delay.
   3.9 Call Demux_control to set FPS.
7. Call Demux_control to get the length of the input.
9. Call Starttitle to display the caption.
11. Call Loadsubtitles to load the caption.
13. Call loadslaves, meaning unknown.
15. Call Initprograms, set es_out and decoder related.
    8.1 Call Updateptsdelay to calculate the correct pts_delay value.
    8.2 Sout related optional, temporarily not analyzed.
    8.3 Call Es_out_setmode, set Es_out MODE to Es_out_mode_auto.
    8.4 Call Demux_control,demux_set_group/set_es only a hint for demuxer (mainly DVB) with demux_set_group instruction to allow not reading Eve Rything.
17. Continue 8.3, the actual call esoutcontrollocked into the case Es_out_set_mode branch.
    9.1 Set the es_out_sys_t b_active and I_mode.
    9.2 Call the Esoutselect method to select a es_out based on the specified module.
    9.3 Enter the Es_out_mode_auto branch in the Esoutselect method, call the Esselect method further, and then call the Escreatedecoder method to create the decoder.
    9.3.1 calls Input_decodernew to create a new decoder.
    9.3.2 if a cache is required, call input_decoderstartwait to send a signal and start the thread wait.
    9.3.3 calls Esoutdecoderchangedelay to set decode delay.
19. Continue 9.3.1 into the Decoder_new method.
    10.1 Call Createdecoder to create the decoder configuration structure body.
    10.1.1 calls Vlc_custom_create to create a VLC object (decoder_t).
    10.1.2 New decode FIFO.
    10.1.3 calls Module_need to load the appropriate decoding module.
    10.1.3.1 calls the Opendecoder method of the Decode module, taking the Codec/avcodec module as an example.
    10.1.3.2 calls the Getffmpegcodec method determine codec type (source description).
    10.1.3.3 calls the Vlc_init_avcodec method to dissolve the code in the initial environment.
    10.1.3.4 calls Avcodec_find_decoder to set Avcodec.
    10.1.3.5 calls Avcodec_alloc_context3 to assign a avcodeccontext.
    10.1.3.6 calls the Init*dec series initial dissolve code environment.
    10.1.4 initializes the other members of the decoder_t struct.
    10.2 Call Vlc_clone to create the decoding thread decoderthread.
21. Continue 10.1.3.5, take Initvideodec as an example.
    12.1 allocates memory for the decoder_sys_t structure.
    12.2 Set the related callback method.
    12.3 Set the decoding thread type.
    12.4 Call Ffmpeg_initcodec to initialize Extradata related data.
    12.5 Call the Openvideocodec method, set the decoding length width and adoption rate, and further call Avcodec_open2 to open codec.
23. Set the thread priority as needed.
25. Set meta-correlation.
27. Initialization is complete, setting the input status to playing_s.

(4) Playback input

Mainloop (SRC/INPUT/INPUT.C)

2. Call Mainloopdemux to access demuxer to Demux data.
4. The Demux method that is set when loading the Demux module is further called, and the Demux method (MODULE/DEMUX/AVFORMAT/DEMUX.C) is actually called with the Avformat/demux module as an example.
   2.1 Call Av_read_frame to read a frame of data.
   2.2 When the error is read, the block_t structure is allocated memory and the frame is copied from the Avpacket to the block_t structure.
   2.3 If the frame is an I-frame, set the I-frame Peugeot bit.
   2.4 Timestamp processing related, not in-depth analysis.
   2.5 call Es_out_control to set up PCR as needed without deep analysis.
   2.6 Call Es_out_send to send this frame of data to es_out.
   2.7 Call Av_free_packet to release this frame of data.
6. After calling Es_out_send, the Esoutsend (src/input/es_out.c) method is actually called.
   3.1 Call Stats_update update related state, specific unknown analysis.
   3.2 Set the read-ahead correlation, set the BLOCK_FLAG_PREROLL flag bit if pre-read is required, and the PTS to the data is less than the time required for pre-reading.
   3.3 Check sout mode, specific sync and async mode, the similarities and differences are not detailed analysis.
   3.4 If the record is set, the data DUP is sent decoder.
   3.5 Call Input_decoderdecode to send the block_t data to the decode FIFO.
   3.5.1 determines the control speed thread waits for relevant information, not detailed analysis.
   3.5.2 If the decode FIFO exceeds the maximum length, the Reset decode FIFO is emptied.
   3.5.3 calls Block_fifoput to press the block_t data into the decode FIFO and notifies the read thread.
   3.6 Format change judgment processing related, not detailed analysis.
   3.7 subtitle Processing related, not detailed analysis.
8. 3.5 continues to enter decode read thread, which is Decoderthread (SRC/INPUT/DECODER.C).
   4.1 calls the Block_fifoget method to fetch data from the decode FIFO.
   4.2 is based on certain conditions, and sending stops waiting for messages to other threads without detailed analysis.
   4.3 calls the Decoderprocess method to start decode a block.
   4.4 Determine the format of the input stream, call the different methods, here in the video stream as an example, call the Decoderprocessvideo method. The
   4.5 Packetizer is related to deep analysis, and the Decoderdecodevideo method is further called in the Decoderprocessvideo method.
10. Continue 4.5 Call Pf_decode_video, here take the decoder of the Avcodec module, for example, the Decodevideo (modules/codec/avcodec/video.c) method, in which the real decoding begins.
    5.1 If the stream information obtained in Demux contains a new extradata, and the original Extradata data is empty, the FFMPEG_INITCODEC initialization codec is called, and if B_delayed_open is true, Then call Openvideocodec to reopen codec.
    5.2 Call Av_init_packet initial dissolve code packet.
    5.3 Call Avcodec_decode_video2 to decode the data.
    5.4 Call Av_free_packet to free memory.
    5.5 Calculates the PTS value and returns the decoded data.
    5.6 If opaque is empty, call the Ffmpeg_newpictbuf method to create a new picture buffer. The specific call to the callback pointer pf_vout_buffer_new points to Vout_new_buffer, which further calls Input_resource_requestvout final call voutcreate.
    5.6.1 calls Vlc_custom_create to create a VLC object (vout_thread_t).
    5.6.2 calls Spu_create to initialize the sub picture unit.
    5.6.3 calls Vlc_clone to create an output thread (SRC/VIDEO_OUTPUT/VIDEO_OUTPUT.C).
    The 5.6.4 output thread loops through the Vout_control_pop, first entering the Threadcontrol method, executing the threadstart direction, and creating the picture FIFO (P->DECODER_FIFO).
12. After Pf_decode_video returns, the decoded data is saved in P_pic, and the Decoderplayvideo method is further called, in which vout_putpicture the decoded data is pressed into the picture FIFO.
14. When there is data in the picture FIFO, the vout thread calls the Threaddisplaypreparepicture method in Threaddisplaypicture.
    7.1 Call Picture_fifo_pop to get the decoded data from the picture FIFO.
    7.2 If the delay is too large and the delay drop frame is set, the frame data is discarded.
16. Call Threaddisplayrenderpicture to display the image.

0X06 Summary

The process analysis of VLC is mainly carried out by tracking the flow of data. The analysis of the final display section is not enough, and many details are not yet in-depth.

Reference:

1. http://blog.csdn.net/tx3344/article/details/8517062
2. http://my.oschina.net/xiaot99/blog/197555

Reprint: VLC Architecture and process analysis