Mediastreamer2-multimedia stream engine

Original article: http://www.linphone.org/eng/documentation/dev/mediastreamer2.html

Mediastreamer2 is a powerful and compact stream engine specially developed for Audio/Video Phone applications.
This library provides processing for all received and sent Multimedia Streams in Linphone, including audio/video capturing, encoding, decoding, and rendering.

Features

• Read/write from to an ALSA device, an OSS device, a Windows waveapi Device
• Send and receive RTP packets
• Encode and decode the following formats: speex, g711, GSM, ilbc, Amr, h263, theora, MPEG4, and hwing.
• Read and Write from/to a WAV file
• Read YUV pictures from a webcam (provided that it has video4linux V1 or V2 driver)
• Display YUV pictures (using SDL library or native APIs on Windows)
• Dual tones generation
• Echo cancelation, using the extraordinary echo canceler algorithm from the speex Library
• Audio Conferencing
• Audio parametric equalizer using a FIR Filter
• Volume control, automatic gain control

Mediastreamer2 can be expanded using dynamic plug-ins. Currently, available plug-ins include h264, ilbct, and Amr.

Portability

• Linux/x86 Linux/x86_64
• Embedded Linux: arm and Blackfin
• Windows XP, Vista and 7
• Mac OS X
• Google Android

Design Concept
Each processing entity is encapsulated in the msfilter object. Msfilter uses inputs and outputs to connect to other msfilters.
A small example for help:

• Msrtprecv is a msfilter that receives RTP packets from the network. After unpacking, the data is sent to the output.
• The processing of msspeexdec depends on its input. If the input is a speex encoded package, it is decoded and the result is sent to the output.
• Msfilerec processing also depends on its input. If the input is a 16-bit PCM data, it is saved as a WAV file.

Msfilters can be connected to form a filter chain. If we connect the above three filters together, we can get a processing chain, receive the RTP packet, decode and write the result to the wav file.
A Media Processing Task schedules images using the "msticker". A thread processes data in the msfilter chains every 10 ms. Several msticker pairs can exist at the same time, such as processing audio filters and processing video filters.

Easy to use
If you want to create an audio/video stream, you can use the simple APIs defined in audiostream. h and videostream. h to create an audio/video stream.
If your purpose is to add new features, you will be happy to find that implementing mediastreamer2 filter is very easy, and mediastreamer2 filter is a very lightweight encapsulation.

Document
Mediastreamer2 doxygen documentation, http://download-mirror.savannah.gnu.org/releases/linphone/mediastreamer/doc/modules.html

Suitable for Embedded Systems

• Mediastreamer2 is lightweight. For example, on Linux/x86 platforms, the full functionality of shared libraries is only about 800 K.
• Pure C Development
• You only need ortp and libc as the minimum dependent libraries, and add others (FFMPEG, speex, ALSA...) as needed.
• The plug-in mechanism is provided, and mediastreamer2 can be easily expanded, such as codecs.

Download
Mediastreamer2 is
You may also use git to obtain the latest source code:
Git clone git: // git.linphone.org/mediastreamer2.git

Use the GPL license agreement