GStreamer is the programming framework (framework) used to build streaming media applications in the GNOME desktop environment, the goal of which is to simplify the development of audio/video applications that can now be used for processing such as MP3, Ogg, MPEG1, MPEG2, AVI, Quicktime Multimedia data in a variety of formats.
I. Basic Concepts GStreamer as a recommended streaming media application framework for the GNOME desktop environment, using a plug-in (plugin) and pipeline (pipeline) architecture, all of the functional modules in the framework are implemented as pluggable components (component), And when the need to be able to easily install to any one pipe, because all plug-ins through the pipeline mechanism for a unified data exchange, it is easy to use the various plug-ins have been "assembled" a full-featured multimedia applications.
1.1 Component Processing
For programmers who need to apply the GStreamer framework, gstelement is a concept that must be understood because it is the basic component of the pipeline and the basis of all the components available in the framework, and it is no wonder that most of the functions in the GStreamer framework involve The operation of the Gstelement object. From the point of view of GStreamer itself, gstelement can be described as a black box with a specific attribute, which interacts with the outside world through a link point, characterizing its features or functions to the rest of the frame.
According to their respective functional differences, GStreamer gstelement subdivided into the following categories:
-The source element data sources component has only output, it can only be used to generate data for pipeline consumption, and can not do any processing of data. An example of a typical data source component is an audio capture unit that reads the original audio data from the sound card and then supplies it to other modules as a data source.
-Filter element filter elements have both input and output terminals, which obtain the corresponding data from the input, and pass to the output end after special processing. An example of a typical filter element is an audio coding unit, which obtains audio data from the outside world, encodes it according to a specific compression algorithm, and finally provides the encoded result to other modules.
-The Sink element receiver component has only input, it has the ability to consume data, and is the terminal of the entire media pipeline. An example of a typical receiver component is the audio playback unit, which is responsible for writing the received data to the sound card, which is usually the last part of the audio process.
Figure 1 will help you better understand the differences between data source components, filter components, and receiver components, and it is also easy to see how they fit together to form pipelines:
Figure 1
It is important to note that the specific form of the filter element is very flexible, GStreamer "does not strictly specify the number of input and output terminals, in fact they can be one or more." Figure 2 is the basic structure of an AVI separator that separates the input data into separate audio and video information, and the filter element that implements the function obviously has only one input, but needs two output terminals.
Figure 2
The only way to create a Gstelement object in your application is by using the factory object gstelementfactory. Because the GStreamer framework provides multiple types of Gstelement objects, it provides a number of types of gstelementfactory objects that are differentiated by a specific factory name. For example, the following code obtains a factory object named Mad by using the Gst_element_factory_find () function, which can then be used to create a corresponding MP3 decoder component:
Once you have successfully obtained the factory object, you can then create a specific Gstelement object by using the Gst_element_factory_create () function, which has two arguments when it is called, the factory object that you need to use, and the name of the component you are about to create. A component name can be obtained by querying, or you can generate a default component for a factory object by passing in a null pointer (NULL). The following code demonstrates how to use the Acquired factory object to create a MP3 decoder component named decoder:
Gstelement *element; element = Gst_element_factory_create (factory, "decoder");
When the gstelement you create is no longer in use, you must also call the Gst_element_unref () function to free the memory resources it occupies:
Gst_element_unref (Element);
GStreamer uses the same mechanism as GObject to manage properties, including queries (query), settings (set), and read (get). All Gstelement objects need to inherit the most basic property of name (name) from their parent object Gstobject, because like Gst_element_factory_make () and Gst_element_factory_ A function such as create () uses the name attribute when creating factory objects and component objects, by invoking the Gst_object_set_name () and Gst_object_get_name () functions to set and read the name properties of the Gstelement object.
1.2 Liner Treatment
Pad (PAD) is another basic concept introduced in the GStreamer framework, which refers to the connection channel between a component (element) and the outside world, and for a particular component in the frame, the type of media it can handle is exposed to other components through a gasket. After the Gstelement object is successfully created, you can obtain the specified liner for the component by Gst_element_get_pad (). For example, the following code returns a liner named src in an element element:
You can also use the gst_element_get_pad_list () function to query for all gaskets in the specified component, if necessary. For example, the following code prints the names of all the liners in the element symbol:
As with components, the name of the pad can be set or read dynamically, by invoking the Gst_pad_get_name () and Gst_pad_set_name () functions. All components of the liner can be subdivided into the input pad and output liner two, which can only receive data input pad, but can not produce data, and output liner is just the opposite, can only produce data but can not receive data, using function gst_pad_get_direction () can get the type of the specified gasket. All gaskets in the GStreamer frame must be attached to a component, calling Gst_pad_get_parent () to obtain the component that the specified liner belongs to, and the return value of the function is a pointer to the gstelement. A gasket can be viewed as a spokesperson for a component in a way that is responsible for describing the capabilities of the component to the outside world. The GStreamer framework provides a unified mechanism for gaskets to describe the capabilities of the components (capability), which are implemented with the help of a data structure _gstcaps:
struct _gstcaps {Gchar *name;/* The name of this caps/* guint16 ID;/* type ID (major type) */Guint refcount; * CA PS are refcounted * * Gstprops *properties; /* Properties for this capability * * Gstcaps *next; /* Caps can be chained together */};
The following is a description of the capabilities of the mad element, and it is not difficult to see that the component actually contains sink and SRC two gaskets, and each liner has specific functional information. The liner, named Sink, is an input to the mad component that accepts media data of MIME type Audio/mp3, plus layer, bitrate, and framed three properties. The liner named SRC is the output of the mad component, which is responsible for generating MIME type Audio/raw media data, as well as the format, depth, rate, and channels properties.
To be exact, each liner in the GStreamer framework may correspond to multiple capability descriptions that can be obtained through function gst_pad_get_caps (). For example, the following code outputs the names of all the capability descriptions in pad pads and their MIME types:
Gstcaps *caps; Caps = gst_pad_get_caps (PAD); G_print ("Pad name is:%sn", gst_pad_get_name (PAD)); while (Caps) {g_print ("Capability name are%s, MIME type is%sn", Gst_caps_get_name (CAP), Gst_caps_get_mime (CAP)); Caps = caps->next; }
1.3 Case Cabinet
A box cabinet (bin) is a container element in a GStreamer frame that is usually used to hold other component objects, but because it is also a Gstelement object, it can actually be used to accommodate other Cabinet objects. A cabinet can combine multiple components that need to be processed into a single logical component, and it is easy to use it to construct more complex pipelines because the components in the cabinet are no longer required to operate individually. Another advantage of using cabinets in the GStreamer framework is that it will try to optimize the flow of data, which is attractive for multimedia applications.
Figure 3 depicts the typical structure of a cabinet in a GStreamer frame:
Figure 3
There are two main types of cabinets used in the GStreamer application:
-Gstpipeline pipeline is the most commonly used container, for a GStreamer application, its top cabinet must be a pipe.
-The role of the gstthread thread is to provide the ability to synchronize, if the GStreamer application requires strict audio and video synchronization, generally need to use this type of cabinet.
The GStreamer framework provides two ways to create a cabinet: one with a factory approach and the other with a specific function. The following code demonstrates how to use the factory method to create a thread object and how to use a specific function to create a pipe object:
Gstelement *thread, *pipeline; Creates a thread object and assigns it a unique name. Thread = Gst_element_factory_make ("thread", NULL); Creates a specific pipe object, based on the name given. Pipeline = gst_pipeline_new ("Pipeline_name");
After the cabinet is successfully created, you can call the Gst_bin_add () function to add a component that already exists:
Gstelement *element; Gstelement *bin; Bin = gst_bin_new ("Bin_name"); element = Gst_element_factory_make ("mpg123", "decoder"); Gst_bin_add (Gst_bin (bin), Element);
It is also easy to find a specific component from a cabinet, which can be achieved by using the Gst_bin_get_by_name () function:
Gstelement *element; element = Gst_bin_get_by_name (Gst_bin (BIN), "decoder");
Because a cabinet in the GStreamer frame can be added to another cabinet, it is possible that the case is nested, and the Gst_bin_get_by_name () function searches for nested cabinets as they look for components. Components are added to the cabinet and can be removed from the box when needed, by invoking the Gst_bin_remove () function:
If you take a closer look at the case described in Figure 3, you will find that it does not have its own input pad and output liner, so it is obviously impossible to interact with other components as a logical whole. To address this problem, GStreamer introduced the concept of the Elf liner (Ghost pad), which is selected from the liner of all components in the cabinet, and usually selects both the input and output liners, as shown in Figure 4:
Figure 4
A cabinet with an elf liner behaves exactly like a component in its behavior, and all components have attributes that all have, and all the operations that can be done on the component can also be done against the cabinet, so you can use this type of cabinet in the GStreamer application as if you were using a component. The following code demonstrates how to add an elf liner to a cabinet:
Second, component connection After the introduction of the concept of components and gaskets, the GStreamer processing of multimedia data becomes very clear: by connecting the different components of the liner in order to form a media processing pipeline, so that the data flow through the pipeline can be normal processing of the various components, the final implementation of specific multimedia functions.
Figure 1 depicts a very simple pipe, it consists of three basic components: The data source component is responsible for generating data only, and its output liner is connected with the input gasket of the filter element, and the filter element is responsible for obtaining data from its own input gasket and after a specific processing, The result is passed through the output liner to the receiver component connected to it; The receiver component is only responsible for receiving data, its input liner is connected with the output gasket of the filter element, and is responsible for handling the final result accordingly.
The components in the GStreamer frame are connected by their respective gaskets, and the following code demonstrates how to connect two components through a gasket and how to disconnect them when needed:
If you need to establish a connected component with only one input pad and one output pad, it is simpler to invoke the Gst_element_link () function to establish a connection between them directly or to call the Gst_element_unlink () function to disconnect them:
third, Component status When components in the GStreamer frame are connected through a pipe, they start their own processing process, typically experiencing multiple state transitions, each of which will be in one of the following four states at a given moment:
-NULL This is the default state for all components, indicating that it has just been created and has not begun to do anything.
-READY indicates that the components are ready and ready to begin processing processes.
-paused indicates that the component temporarily stops processing data for some reason.
-PLAYING indicates that the component is processing data.
All components start with a null state and then undergo transitions between null, READY, paused, PLAYING, and so on. The current state of the component can be toggled by calling the Gst_element_set_state () function:
Gstelement *bin; /* Create the component and connect it to the box bin/gst_element_set_state (bin, gst_state_playing);
By default, the pipe and all the components it contains are in a NULL state after it is created, and they do nothing at this time. After the pipe has been used, do not forget to switch the state of the pipe back to NULL so that all components contained therein can have the opportunity to release the resources they are consuming.
The pipe's true processing process begins when it is switched to the READY state for the first time, and the pipeline and all the components it contains are initialized to prepare for the data processing that is about to be performed. For a typical component, the actions to be performed in a READY state include opening media files and audio devices, or attempting to establish a connection with a media server located remotely.
Once the pipeline in the READY state is switched to the PLAYING state, the multimedia data that needs to be processed begins to flow through the entire pipeline and is then processed by the various components contained in the pipeline, thus eventually realizing a certain multimedia function that the pipeline has predefined. The GStreamer framework also allows the pipe to be switched directly from a NULL state to a PLAYING state without passing through the intermediate READY state.
A pipe that is in the playback state can switch to the paused state at any time, temporarily stop the flow of all data in the pipeline, and can switch back to the PLAYING state again when needed. If you need to insert or change a component in a pipe, you must first switch it to a paused or NULL state, and the component will not release the resource it occupies when it is in the paused state.
Iv. Implementation of MP3 player After understanding some basic concepts and processes, let's look at how to implement a simple MP3 player using the components provided by the GStreamer framework. The structure described in Figure 1 can easily be mapped to a MP3 player, where the data source component is responsible for reading data from the disk, the filter component is responsible for decoding the data, and the receiver component is responsible for writing the decoded data to the sound card.
Like many other GNOME projects, GStreamer is also implemented in C language. If you want to apply the various features provided by GStreamer in your program, you must first call Gst_init () in the main function to complete the initialization to pass the parameters that the user entered from the command line to the GStreamer function library. The initialization of a typical GStreamer application is as follows:
#include int main (int argc, char *argv[]) {gst_init (&ARGC, &ARGV); /* ... */ }
Next, you need to create three components and connect to pipes, and since all GStreamer components have the same base class Gstelement, you can define them in the following ways:
Pipelines are used to hold and manage components in the GStreamer framework, the following code creates a new pipeline named pipeline:
/* Create a new pipe to hold the component * * pipeline = gst_pipeline_new ("Pipeline");
The data source component is responsible for reading data from the disk file, which has a property named location that indicates the location of the file on disk. Use the standard GObject property mechanism to set the appropriate properties for a component:
/* Create a Data Source component/* Filesrc = Gst_element_factory_make ("Filesrc", "Disk_source"); G_object_set (G_object (FILESRC), "location", argv[1], NULL);
The filter element is responsible for the decoding of the data in the MP3 format, the simplest way is to install the Mad plug-in, with which to complete the corresponding decoding work:
Because the thread is not used, it is necessary to determine when the process of the pipeline will end by continually invoking the Gst_bin_iterate () function:
while (Gst_bin_iterate (Gst_bin (pipeline));
As long as new events are generated within the pipeline, the Gst_bin_iterate () function returns TRUE, and the function returns FALSE only when the entire processing is finished, terminating the pipeline and releasing the resource that is occupied:
The source code for the MP3 player implemented with GStreamer is as follows:
#include int main (int argc, char *argv[]) {gstelement *pipeline, *filesrc, *decoder, *audiosink; Gst_init (&ARGC, &ARGV); if (argc!= 2) {g_print ("Usage:%s n", argv[0]); Exit (-1); /* Create a new pipe/* pipeline = gst_pipeline_new ("Pipeline"); /* Generate components for reading hard disk data * * FILESRC = Gst_element_factory_make ("Filesrc", "Disk_source"); G_object_set (G_object (FILESRC), "location", argv[1], NULL); /* Create decoder component * * decoder = Gst_element_factory_make ("Mad", "decoder"); /* Create audio playback Components * * Audiosink = Gst_element_factory_make ("Osssink", "Play_audio"); /* Add the generated components to the pipe/Gst_bin_add_many (Gst_bin (pipeline), FILESRC, decoder, Audiosink, NULL); /* Connect each component * * Gst_element_link_many (FILESRC, decoder, Audiosink, NULL); /* Start playing * * Gst_element_set_state (pipeline, gst_state_playing); while (Gst_bin_iterate (Gst_bin (pipeline)); /* Stop pipeline process/gst_element_set_state (pipeline, Gst_state_null); /* Release Occupied resources * * GST_OBJECT_UNREF (Gst_object (pipeline)); Exit (0); }
v. Summary With the continuous popularization of GNOME desktop environment, GStreamer as a powerful multimedia application development framework, has begun to receive more and more people's attention. Gstreamer is designed with a very flexible architecture, and provides a number of predefined media processing modules, thus greatly simplifying the difficulty of developing multimedia applications under Linux.
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.
