Get started with mediastreamer2-data structures and common functions

Mediastreamer2 is an open-source Streaming Media Processing framework that abstracts and schedules media transmission, processing, storage, and playback processes.
Two important concepts in mediastreamer2: Filter and graph. Some words become more appealing after being translated in Chinese. In fact, many audio and video frameworks or software also use the word filter.

1) filter: A filter is a component used to process media data in mediastreamer2. One filter can have multiple input and output values, including inputs and outputs.
It is actually a buffer queue.

Mediastreamer2 uses the msfilter object to abstract each step of the processing process.

Each msfilter object has a corresponding descriptor, represented by the msfilterdesc descriptor of the data structure, this structure describes the IDs, names, number of input and output scripts, initialization functions, preprocessing, and processing functions of the msfilter object. Each msfilter object also has its own msqueue (inputs, outputs ).

Mediastreamer2 media streams can be sent and received through RTP, received and sent by media streams, and played, encoded, and decoded by filters. For example
Msrtprecv is a msfilter object of mediastreamer2. It indicates receiving data from the network using the RTP protocol.
Data is output to the buffer queue.
In fact, filters communicate with each other by sharing message queues.

2) graph: graph is a filter manager that connects multiple media streams from one fiilter output to the next one through system management and scheduling.
Input of the filter.
Mediastreamer2 provides the ms_filter_link function to link the msfilter object to form a graph with multiple filters,
Ms_filter _ Link (first _ filter, output _ pin, second _ filter, input _ pin );
As shown above, ms_filter_link can share data between two msfilter objects.

After creating a graph, you need to schedule the msfilter object, which is completed by the msticker object:
Msticker ticker = ms_ticker_new ();
Ms_ticker_attach (ticker, msfilter *);

Mediastreamer2's audio processing mechanism:
Receiving link:
Sending link: Sound Card acquisition (filter) -- audio processing (filter) -- audio encoding -- RTP packet sending (filter)
RTP packet receiving (filter) -- audio decoding (filter) -- audio processing (filter) -- tee (filter) -- sound card playing (filter)

|

---- WAV save (filter)

In the source code of mediastreamer2, audio streams are encapsulated in the audio_stream_start_full function,
Audio_stream_start_full: The main goal of this function is to complete the initialization of each filter in audiostream and set
Sound card read/write, RTP data transmission/receiving, encoding, encoder process, first load the RTP payload_type, and jitter compensation, at the same time call
Ms_filter_call_method is used to initialize other attributes of RTP, link the filter to two audio processing chains for receiving and sending, and start ticker.
To enable the audio link to work.
Videostream is completed by the video_stream_start function.

Audio_stream_new: Creates a new audiostream and initializes rtpsession and filter rtpsend.

Initialize the filter: msfilter * ms_filter_new (msfilterid ID) function. This function initializes a filter through the msfilterid parameter and returns the new filter to the caller through the function.

Ms_filter_create_decoder: Use the autostream or videostream parameters to find the corresponding payload in the rtpsession, and then create a decoder Filter Based on the MIME type in the payload. First, find msfilterdesc that complies with ms_filter_decoder and mime, and bind msfilter through ms_filter_new_from_desc.

Ms_connection_helper_start (msconnectionhelper * helper) creates a helper, and helper is a struct.
Ms_connection_helper_link (msconnectionhelper * helper, msfilter * filter, int inpin, int outpin) connects the filter to the tail end of the link, in fact, this function is an encapsulation of ms_filter_link.

Ms_filter_link (msfilter * F1, int pin1, msfilter * F2, int pin2) uses the pin1 interface of F1 as the output port and the pin2 interface of F2 as the input interface for link.

Filter void audio_stream_record (audiostream * St, const char * Name) function sets the path of the recording file. This function is for int ms_filter_call_method (msfilter * F, unsigned int ID, void * Arg) function packaging.
The Int ms_filter_call_method (msfilter * F, unsigned int ID, void * Arg) function uses the second parameter unsigned int ID to find the function mounted when the method is executed. The third parameter, is the parameter passed in to the Mount function.
The recording file of the audio_stream_record (stream, "/tmp/1.wav") link is located at/tmp/1.wav.