OpenWrt GStreamer Example Learning Note (six. GStreamer pads and its functions)

One: Overview

As we saw in the elements chapter, pads is an element external interface. The data stream from one element of the source pad to another element of the sink pad. The pads function (capabilities) determines the type of media that an element can handle.

The type of a pad is determined by 2 characteristics: its data-oriented (direction) and its timeliness (availability). As we mentioned earlier, GStreamer defines 2 kinds of pads data-oriented: source pad and sink pad. Pads's data-driven terminology is defined from the perspective of the element inside:

element receives data through their sink pad and outputs data through their source pad. The timeliness of pads is much more complex than pads's data-orientation. A pads can have three types of timeliness: permanent (always), on-model (sometimes), request-based. Three the meaning of timeliness as the name implies: Permanent pads will always exist, the random type of pads only in certain conditions only exist (the random disappearance of pads is also a random type), the request-type pads only when the application explicitly issued the request.

1). dynamic (Random) pads

Some element does not immediately produce all the pads it will use when it is created. This can happen, for example, in an element of Ogg demuxer. This element will read the Ogg stream, and whenever it detects some metadata flow in the Ogg stream (for example, Vorbis,theora), it creates a dynamic pads for each metadata stream. Similarly, it will also delete the pads when the stream terminates. Dynamic pads can play a significant role in the demuxer element.

Running Gst-inspect Oggdemux only shows a pads in the component: a sink pad called ' sink ', and the rest of the pads are ' dormant ' and you can get from the pad template in the "Exists: Sometimes "property to see this information. The pad will be based on the type of ogg file you are playing, and recognizing this is especially important for you to create a dynamic pipeline. When pads creates a random (sometimes) pads by its random type (sometimes) pads template, you can know that signal was created by binding a signal processor (handler pads) to the element. The following code demonstrates how to do this:

The sink pads called ' sink ', the rest of the pads are in ' sleep ', and it is obvious that this is pads "sometimes exists" feature. Pads will be based on the type of ogg file you are playing, which is especially important when you are ready to create a dynamic pipeline, and when element creates a "sometimes present" pad, you can trigger a signal processor (signal handler) on the element. To know that pads was created.

#include <gst/gst.h>

static void Cb_new_pad (Gstelement *element, Gstpad *pad, gpointer data) {

Gchar *name;

name = Gst_pad_get_name (PAD);

G_print ("A new pad%s was created\n", name);

G_free (name);

/* Here, your would setup a new pad link for the newly created pad */

[..]

}

int main (int argc, char *argv[])

{

Gstelement *pipeline, *source, *demux;

Gmainloop *loop;

/* init */

Gst_init (&ARGC, &ARGV);

/* Create elements */

Pipeline = gst_pipeline_new ("My_pipeline");

Source = Gst_element_factory_make ("filesrc", "source");

G_object_set (source, "location", argv[1], NULL);

Demux = Gst_element_factory_make ("Oggdemux", "demuxer");

/* would normally check that the elements were created properly */

/* put together a pipeline */

Gst_bin_add_many (Gst_bin (pipeline), source, Demux, NULL);

Gst_element_link_pads (source, "src", Demux, "sink");

/* Listen for newly created pads */

G_signal_connect (Demux, "pad-added", G_callback (Cb_new_pad), NULL);

/* Start the pipeline * *

Gst_element_set_state (Gst_element (pipeline), gst_state_playing);

loop = G_main_loop_new (NULL, FALSE);

G_main_loop_run (loop);

[..]

}

2). Request Pads

Element can also have request pads. This pads is not created automatically, but is created on request. This is useful in a multiplexed (multiplexers) type of element. such as aggregators and tee element.

The aggregators element can combine multiple input streams into one output stream; The tee element is just the opposite, it has only one input stream, and then sends the data stream to different output pads according to the request. As long as the application requires another data stream, it can simply request from the tee element to an output pads.

The following code shows how to request a new output pad in a "tee" element:

static void Some_function (Gstelement *tee)

{

Gstpad * PAD;

Gchar *name;

Pad = Gst_element_get_request_pad (tee, "src%d");

name = Gst_pad_get_name (PAD);

G_print ("A new pad%s was created\n", name);

G_free (name);

/* Here, you would link the pad */

[..]

/* and, after doing., free reference */

Gst_object_unref (Gst_object (PAD));

}

The Gst_element_get_request_pad () method can get a pads from an element, pads based on the name of the pads template (pad template). It is also possible to request a pads that is compatible with other pads templates, which is important in some cases.

For example, when you want to connect an element to a multiplexed element, you need to request a compatibility pad. The Gst_element_get_compatible_pad () method can be used to get a pad with compatibility.

The following code will request a compatible pads from a Ogg-based element with multiple input pads.

static void Ink_to_multiplexer (Gstpad *tolink_pad, gstelement *mux)

{

Gstpad *pad;

Gchar *srcname, *sinkname;

SrcName = Gst_pad_get_name (Tolink_pad);

Pad = Gst_element_get_compatible_pad (MUX, Tolink_pad);

Gst_pad_link (Tolinkpad, pad);

Sinkname = gst_pad_get_name (PAD);

Gst_object_unref (Gst_object (PAD));

G_print ("A new pad%s is created and linked to%s\n", SrcName, Sinkname);

G_free (Sinkname);

G_free (SrcName);

}

II: Performance of (Pads)

Since pads plays a very important role in an element, there is a term to describe the flow of data that can be passed through pads or currently through pads. The term is function (capabilities).

The functions of pads (capabilities) are associated with pads templates (Pad templates) and pads instances. For the pads template, the pads function (capabilities) describes the type of media that the pads allows to pass through when a pads is created through the pads template.

For pads instances, a feature can describe all of the media types that may be passed through the pads (typically a copy of the functionality of the pads template to which the pads instance belongs), or a streaming media type that is currently passing through the pads. In the former case, the pads instance has not yet had any data flow in it.

1). decomposition function

The function of pads is described by the Gstcaps object. A Gstcaps object will contain one or more gststructure. A gststructure describes a media type. A data-flowing pad (negotiated pad) has a feature set (capabilities set), each of which contains only one gststructure structure. Only fixed values are included in the structure. However, the above constraints are not valid for pads (unnegotiated pads) or pads templates that have not yet been circulated.

Here's an example where you can see some of the features of the "Vorbisdec" component by running Gst-inspect Vorbisdec. You may see 2 pads:source pad, sink pad, 2 pad of timeliness are permanent, and each pads has a corresponding function description. The sink pads receives Vorbis encoded audio data, and its mime-type is displayed as "Audio/x-vorbis". The source pads can send the decoded audio data sample (raw audio samples) to the next element, whose mime-type is displayed as "Audio/x-raw-int". Some other features are included in the source Pad feature Description: Audio samplerate, number of channels, and some information that you may not be too concerned about.

Pad Templates:

src Template: ' src '

Availability:always

Capabilities:

Audio/x-raw-float

Rate: [8000, 50000]

Channels: [1, 2]

endianness:1234

Width:32

buffer-frames:0

SINK Template: ' SINK '

Availability:always

Capabilities:

Audio/x-vorbis

2). Characteristics and values

Attributes (properties) are used to describe the additional information in a feature (note: Information other than the type of data flow). An attribute consists of a keyword and a value. Here are some of the types of values:

The basic type, which covers almost all the gtype types in glib. These types indicate a definite, non-dynamic value for each attribute (properties). The examples are as follows:

Integral type (g_type_int): A definite value (relative range value).

Boolean type: (G_type_boolean): TRUE or False.

Floating-point type: (g_type_float): Explicit floating-point number.

String type: (g_type_string): UTF-8 encoded string.

Fractional type: (gst_type_fraction): The fraction of the numerator denominator that is made of integers.

Scope type (range types): A data type that is registered by GStreamer and belongs to Gtypes. It indicates a range value. The range type is typically used to indicate the supported audio sample rate range or the supported video file size range. There are also 2 different types of range values in GStreamer.

Integer range Value (gst_type_int_range): An integer value range is indicated with the maximum and minimum boundary values. For example, the sample rate range for the "Vorbisdec" element is 8000-50000.

Floating-point range Value (Gst_type_float_range): a range of floating-point values is indicated with the maximum and minimum boundary values.

Fractional range Value (Gst_type_fraction_range): A fractional value range is indicated with the maximum and minimum boundary values.

List Type (gst_type_list): You can take any one of the values in the given list. Example: function of a gasket if you want to indicate that it supports data with a sample rate of 44100Hz and 48000Hz, it can use a list data type that contains 44100 and 48000.

Array type (Gst_type_array): A set of data. Each element in the array is the full value of the attribute. The elements in the array must be of the same data type. This means that an array can contain any number of integers, a list of integral types, and a combination of integer ranges. This is true for floating-point numbers and string types, but an array cannot contain both integers and floating-point numbers. Example: For an audio file with more than two channels, the channel layout needs to be specifically specified. (for mono and dual-channel audio files, it is handled by default unless you explicitly indicate the number of channels in the feature). So the channel layout should be stored with an enumerated array type. Each enumeration value represents a loudspeaker position. Unlike the gst_type_list type, the array type is treated as a whole.

3). Purpose of Pads performance

The pads function (capabilities) (Caps) describes the types of data flows between two pads, or the types of traffic they support. Functions are mainly used for the following purposes:

AutoFill (autoplugging): Automatically finds the element that can be connected, based on the function of element. All automatic fillers (autopluggers) are used in this way.

Compatibility detection (compatibility detection): When two pads are connected, GStreamer verifies that they are interacting with the same data flow format. The process of connecting and verifying the compatibility of two pads is called "functional negotiation" (caps negotiation).

Metadata (Metadata): By reading the pads function (capabilities), the application is able to provide information about the media type that is currently playing through the pads. And this information is called metadata (Metadata).

Filtering (Filtering): Applications can limit the media types between the two interactive pads through the pads function (capabilities), which is a set of restricted media types that should be a subset of the set of formats supported by the pads of two interactions. For example, the application can use "filtered caps" to indicate the supported video sizes (fixed or non-fixed) for the two interactive pads. You can insert a capsfilter element into your pipeline and set its pads function (capabilities) property to enable filtering of the pads function (capabilities). The function filter (Caps filters) is typically placed behind some conversion element, forcing the data to be converted to a specific output format at a specific location. These conversion element are: Audioconvert, Audioresample, Ffmpegcolorspace and Videoscale.

I) using the pads function (capabilities) to manipulate meta data

A pads can have multiple functions. A function (Gstcaps) can be represented by an array that contains one or more gststructures. Each gststructures consists of a string of names (such as "width") and corresponding values (types may be g_type_int or gst_type_int_range).

It is worth noting that there are three different pads functions (capabilities) that need to be differentiated: the possible functions of pads (possible capabilities) (usually obtained by pads template gst-inspect), Pads the Allow function (allowed caps) (which is a subset of the functionality of the pads template, depending on the possible functionality of each pair of interaction pads), pads the last negotiation function (lastly negotiated caps) (exact stream or cache format, contains only one structure, And there are no indeterminate variables such as range values or list values.

You can get all the features of a pads feature set by querying the structure of each feature. You can get a function of Gststructure by Gst_caps_get_structure () and get the gststructure number in a Gstcaps object by Gst_caps_get_size ().

Simple pads function (capabilities) refers to only one gststructure, fixed gasket function (capabilities) (fixed caps) means that it has only one gststructure, And there are no mutable data types (like ranges or lists, etc.). There are also two special features-any pads function (capabilities) (any caps) and the function of the empty pads (capabilities) (empty caps).

The following example shows how to extract the width and height information from a fixed video function:

static void Read_video_props (Gstcaps *caps)

{

Gint width, height;

Const Gststructure *STR;

G_return_if_fail (gst_caps_is_fixed (Caps));

str = gst_caps_get_structure (Caps, 0);

if (!gst_structure_get_int (str, "width", &width) | |

!gst_structure_get_int (str, "height", &height)) {

G_print ("No width/height available\n");

Return

}

G_print ("The video size of this set of capabilities is%dx%d\n",

width, height);

}

II). function (capabilities) applied to the filter

Since the pads function (capabilities) is often included in the plug-in (plugin) and is used to describe the type of media supported by pads, the programmer, in order to interact with the plug-in (plugin), especially when using the filter function (filtered caps), A basic understanding of the pads function is often required. When you use the filter function (filtered caps) or the fixed function (fixation), you limit the type of media that is allowed between the pads of the interaction, limiting it to a subset of the media types supported by the interactive pads. You can do this by using the Capsfilter element in your pipeline, and in order to do this, you need to create your own gstcaps. The easiest thing to do here is that you can create your own gstcaps by using the Gst_caps_new_simple () function.

Static Gboolean

Link_elements_with_filter (gstelement *element1, gstelement *element2)

{

Gboolean LINK_OK;

Gstcaps *caps;

Caps = gst_caps_new_simple ("Video/x-raw-yuv",

"Format", GST_TYPE_FOURCC, GST_MAKE_FOURCC (' I ', ' 4 ', ' 2 ', ' 0 '),

"width", g_type_int, 384,

"Height", G_type_int, 288,

"Framerate", Gst_type_fraction, 25, 1,

NULL);

LINK_OK = gst_element_link_filtered (element1, element2, caps);

Gst_caps_unref (Caps);

if (!LINK_OK) {

G_warning ("Failed to link element1 and element2!");

}

return LINK_OK;

}

The code above restricts the data that interacts between the two elment to a specific video format, width, height, and frame rate (two elemment fail if these restrictions are not met). Remember: When you use Gst_element_link_filtered (), GStreamer automatically creates a capsfilter element, adds it to the bins or pipeline, and inserts it into the two element you want to interact with. (You need to be aware of this when you want to disconnect two element connections).

In some cases, when you want to create a more precise set of features with filtered connections between two pads, you can use a leaner function-gst_caps_new_full ():

Static Gboolean Ink_elements_with_filter (Gstelement *element1, gstelement *element2)

{

Gboolean LINK_OK;

Gstcaps *caps;

Caps = Gst_caps_new_full (

Gst_structure_new ("Video/x-raw-yuv",

"width", g_type_int, 384,

"Height", G_type_int, 288,

"Framerate", Gst_type_fraction, 25, 1,

NULL),

Gst_structure_new ("Video/x-raw-rgb",

"width", g_type_int, 384,

"Height", G_type_int, 288,

"Framerate", Gst_type_fraction, 25, 1,

NULL),

NULL);

LINK_OK = gst_element_link_filtered (element1, element2, caps);

Gst_caps_unref (Caps);

if (!LINK_OK) {

G_warning ("Failed to link element1 and element2!");

}

return LINK_OK;

}

4) Elf Pads (Ghost Pads)

Bins has its own pads, this is the origin of "elf pads".

Such as. No gstbin element using the sprite pads

Sprite pads comes from some element in bins, which can also be accessed directly in the bins. The sprite pads is similar to symbolic links in the Unix file system. Using bins, you can use bins as a normal element in your code.

such as. gstbin element using the sprite pads

The sink pad on the leftmost element is also the pads of the entire bins. Because the sprite pads looks no different from other pads, it has similar functions to other pads. So they can be added to any element, not just gstbin.

You can create a ghost pad by using the function gst_ghost_pad_new ():

#include <gst/gst.h>

int main (int argc, char *argv[])

{

Gstelement *bin, *sink;

Gstpad *pad;

/* init */

Gst_init (&ARGC, &ARGV);

/* Create element, add to Bin */

Sink = Gst_element_factory_make ("Fakesink", "sink");

Bin = gst_bin_new ("Mybin");

Gst_bin_add (Gst_bin (BIN), sink);

/* Add Ghostpad */

Pad = Gst_element_get_pad (sink, "sink");

Gst_element_add_pad (Bin, gst_ghost_pad_new ("sink", pad));

Gst_object_unref (Gst_object (PAD));

[..]

}

In the above example, pads not only has the genie pads, but also has a sink pad element named "sink". So this bins can be used as a substitute for that element. You can connect the other element to the bins.

OpenWrt GStreamer Example Learning Note (six. GStreamer pads and its functions)