Playing principles of playbin in gstreamer

1. directory structure
Gstreamer-0.10.21 directory
Core File of GST to implement functions such as the GST component Factory
The components provided by the Plugins gstreamer-0.10.21
Lib cannot be called by the application, but it is the parent class of some components.
Pkgconfig install tools are all the objects to be copied by make install.
Po translation files in various languages
Tests test gstreamer-0.10.21 Components

Directory of GST-plugins-base-0.10.21
Components provided by GST Base
The parent class of the component provided by the GST-libs base, which is used to apply unavailable components.
EXT components requiring additional source code library
Sys system-related components (X11, v4l) do not have these two windows.

2. Number of Elements
Check which components of the GST file to see which files call the gst_element_register
Grep-Nr 'gst _ element_register '-- include = *. C ./

Gstreamer components are registered in a unified manner. Other components are registered separately.
Static struct _ elements_entry _ elements [] = {
{"Capsfilter", maid, maid },
{"Fakesrc", maid, maid },
{"Fakesink", maid, maid },
# Ifdef have_sys_socket_h
{"Fdsrc", maid, maid },
{"Fdsink", maid, maid },
# Endif
{"Filesrc", maid, maid },
{"Identity", maid, maid },
{"Queue", maid, maid },
{"Filesink", maid, maid },
{"Tee", maid, maid },
{"Typefind", maid, maid },
{"Multiqueue", maid, maid },
{Null, 0 },
};

3. playbin (only when playbin plays a video)
Playbin components are integrated components. Many components are used to build factory pipelines.

3.1 component Flowchart
Filesrc --> (typefind) --> decodebin --> | --> Abin (audioconvert --> audioresample --> volume --> osssink)
········································ · | --> Vbin (identity --> ffmpegcolorspace --> videoscale --> fbdevsink)

3.2 code analysis
Call get_element_set_state to set playbin to null, ready, paused, and playing. In ready --> paused (stst_state_change_ready_to_paused), playbin establishes the MPs queue.

Ststst_play_base_bin_change_state
| ---- Setup_source
| ---- Gen_source_element creates the source element, that is, filesrc.
| ---- If (play_base_bin-> suburi) setup_subtitle: configure the subtitle element if suburi attribute of playbin is set
| ---- Maid
| ---- Get_element_factories_from_uri_protocol determine the factory type based on the string header URI protocol search_by_entry. For example, if file: // is set, filesrc is created.
| ---- Maid creates a component (filesrc) based on the determined factory type)

| ---- Whether the analyse_source analysis source is raw data/has dynamic pad/has output pad for corresponding processing
| ---- Group_commit is used in several cases.
| ---- Maid->
Setup_output_pads, that is, setup_sinks of ststplaybin. C.

| ---- You need to call gen_vis_element visually when audio is available.
| ---- You do not need to call gen_audio_element visually when audio is available.
| ---- A video calls gen_video_element.

| ---- Make_decoder
| ---- Maid ("decodebin", null) Create decodebin Element
| ---- Maid (play_base_bin-> source, decoder) link filesrc and decodebin

3.3 playbin directory
Directory is GST-plugins-base-0.10.21/GST/playback/
Originally there were quite a lot of files, but there were only two sets of playbin, one of which was a non-stable version of playbin2. In fact, there were not many files to watch.
Register playbin and playbin2
Playbin uses C files (mainly playbin, playbasebin, and decodebin)
Gstplaybin. c
Gstplaybasebin. c
Gstdecodebin. c
Ststststreaminfo. c
Ststststreamselector. c
Gstplaymarshal. c

Playbin2 uses C files (playbin2, playbasebin, and decodebin are equivalent to playbin2, playsink, and decodebin2)
Gstplaybin2.c
Gstplaysink. c
Gstdecodebin2.c
And the remaining files. The gsturidecodebin. C and gstdecodebin2.c are also unstable versions.

3.4 parent class used by playbin

The parent_class of filesrc is basesrc, which can be seen below,
Ststst_boilerplate_full (ststfilesrc, ststst_file_src, stststbasesrc, ststst_type_base_src,
_ Do_init );

When you call set_state, The change_state of the parent class of the component is called. Basesrc is like the base class of filesrc.

The parent_class of osssink is audiosink. It can be seen from the following,
Osssink_type = g_type_register_static (maid, "gstosssink", & osssink_info, 0 );

Osssink only works on the underlying layer. The audio logic layer is still in audiosink and its parent audiobasesink. They then call the osssink function pointer to perform specific operations on the sound card reading and writing.

3.5 component's parent class
Change_state is distributed to every component in the pipeline, and change_state of parent_class is called in change_state of the component. Therefore, you can check parent_class of playbin.
Struct _ maid {
Gstplaybasebin parent;

Struct _ maid {
Gstpipeline pipeline;

Playbin --> playbasebin --> pipeline is in the MPs queue.

Set_state can be sent to the change_state function of all elements in the pipeline.
The change_state function of each component calls the change_state of parent_class,
Playbin -- (parent) --> playbasebin
Playbasebin -- (parent) --> Pipeline
Pipeline -- (parent) --> Bin
The last level is bin. Check the change_state function of bin,

While (! Done ){
Gpointer data;

Switch (maid (it, & Data )){
Case ststst_iterator_ OK:
{
Gstelement * child;

Child = maid (data );

/* Set state and base_time now */
Ret = maid (bin, child, base_time, current, next );

Extract the components in the bin one by one and send the set_state command, so that all the components will change their status.

Three methods for getting the clock by using stststclock
1 systemclock system clock
2. The clock that the audio device knows Based on the sampling rate
3. Clock information contained in the Network Package

I have only read systemclock, stststclock-> maid-> g_get_current_time-> gettimeofday-> (system call)

Audio/Video time synchronization to system time
It's not who synchronizes audio and video, but who Synchronizes time with the system.
NS... B... EOS NS = new segment
C. running_time = absolute_time-base_time (system)
B. running_time = (B. timestamp-ns. Start)/ns. abs_rate + NS. accum (Component)
Synchronization equals the running_time of B and C, and B waits.
Base_time is sent from the new_segment event as the benchmark. For example, after the seek, it is the new segment.

_ Class_init: pointer to the Klass Function
Gst_base_audio_sink_class_init (gstbaseaudiosinkclass * Klass)
Gobject_class = (gobjectclass *) Klass;
Gstelement_class = (gstelementclass *) Klass;
Stststbasesink_class = (ststbasesinkclass *) Klass;
It is possible to convert a Klass into three types of data structures because the first level is the first member of the data structure. The imported kclass is the largest data structure and can be converted to a small one.

Struct _ maid {
Stststbasesink element;

Struct _ ststbasesink {
Gstelement element;

Struct _ gstelement
{
Ststobject object;

Status Change 
Null --> ready
1 probe Device
2 open device

Ready --> paused
1. Activate pad, return async, and start the stream thread to finish the state change until the sink pad receives the first buffer and blocks it. Then, the status change is completed, get_state can be used to query
2. The pipeline running_time is set to zero.
3. If it is live source, no_preroll is returned and no data is generated (live source generates data sources, such as net and camera, even if it is paused)

Paused --> playing (this state is ignored by most components)
1. The media transcoding queue selects the clock to distribute to each sub-component, that is, the synchronization clock only occurs during playing.
2. The pipeline distributes the base_time calculated by clock running_time to each sub-component (when change_state is used)
3. The sink pad no longer blocks the buffer/event and starts render data.
4. Live source starts to generate data.

Playing --> paused (this state is ignored by most components)
1. If the sink has no buffer at this time, you must wait for the buffer to complete the state change. The above ready-> paused is also the buffer that sink must have before entering paused. In order to playing, it will not let the user wait for a small amount of time.
2. the EOS event is invalid, but it will be repost at playing.
3. The sink removes all waits on the preroll from blocking.
4 live source stop generating data

Paused --> ready
1. The sink unblocks preroll and the component unblocks the device.
2 chain and get_range return wrong_state (buffer-related function)
3. Deactivate and stream thread are disabled.
4 sink forgets all negotiation formats (negotiation caps)

Ready --> null
1 close Device
2. Delete All dynamically created pads