Live555 Study III live555 Basics

Live555 Basics

 

Live555 is a cross-platform C ++ open-source project that provides solutions for streaming media. From today on, we will officially start to go deep into the live555 code.

 

I. Brief Introduction to databases

Live555 contains livemedia, usageenvironment, basicusageenvironment, groupsock library, mediaserver simple server program, and multiple other test demos.

LivemediaLibrary: contains a series of classes that process different encoding formats and encapsulation formats. The base class is medium.

UsageenvironmentLibrary: Environment class, used to output error messages. Most live555 classes contain such object pointers. It contains a pointer to the taskschedule abstract class, which is used for task scheduling. Therefore, all classes that contain the usageenvironment pointer can add themselves to scheduling.

BasicusageenvironmentLibrary: contains the specific environment class and the specific taskscheduler class. Usageenvironment is used to process error messages. The basicusageenvironment class is used to output error messages on the console. Therefore, classes that want to output error messages in other ways can be derived from usageenvironment. The basictaskschedule class inherits from the taskscheduler abstract class to define specific scheduling policies. Any live555-based application needs to define its own basicenvironment and taskscheduler libraries. If you create a window application and redefine taskscheduler, you must integrate it with your own event processing framework in the graphic environment. Basictasksheduler use the select model to obtain and process events. To use a more efficient iocp model, you can define your own basictaskscheduler class. Basictaskscheduler has a loop in which messages in the queue are read and processed cyclically. The entire program based on basictaskscheduler has only one thread driver.

GroupsockLibrary: encapsulate various socket operations for sending and receiving data. It is mainly for multicast, but it can also send and receive data unicast. It only supports UDP and does not support TCP.

MediaserverServer program: This program is implemented using the basicusageenvironment library, so it is a console program. The task scheduling class is the basictaskscheduler class. Therefore, the Select model is used and only one thread processes various events cyclically. If there is time later, the iocp-based mediaserver server program will be implemented.

Other test demos: Client programs implemented based on live555 will be introduced as needed.

 

Ii. Basic Concepts involved

 

1. Souce and sink

Souce: translated as source and source. Indicates the data provider, such as reading data through RTP, reading data through files, or reading data from memory, all of which can be used as Souce.

Sink: translated as sink. Indicates the data flow and consumer. For example, writing a file and displaying it on the screen.

Filter: translated as a filter. You can set filter when data streams flow from Souce to sink for filtering or further processing.

In livemedia, data flows from Souce to sink after one or more filters. On the server, data streams flow from files or devices to the network, while on the client, data streams flow from the network to files or screens.

 

Mediasouce is the base class of all Souce and mediasink is the base class of all sinks.

 

From the number of classes and code scale, livemedia class is the core of the entire live555, which contains dozens of classes with specific encoding and encapsulation formats. Various Souce defined by livemedia are read from files. If you want to transmit real-time streams from devices, you can define your own Souce.

 

2. clientsession

For each client connected to the server, the server creates a clientsession object for it and stores the socket and IP address of the client. At the same time, various response functions are defined in the client to process and respond to various requests of the client. The clientconnection class is added to live555 in the new version. It is used to process commands unrelated to normal playback. For example, the command is not found, the command is not supported, or the media file is not found. When clientconnection processes the describe command, it creates a clientsession object. Other commands are processed in clientsession.

 

3. mediasession, mediasubsession, and track

Live555 uses mediasession to manage a media file containing audio and video. Each mediasession uses a unique identifier of the file name.

Use subsession to manage an audio stream or video stream in mediasession. For ease of writing, we call the audio or video as a media stream in a media file. Therefore, a mediasession can have multiple mediasubsessions, one for managing audio streams and one for managing video streams.

When the client sends describe to the server to query the SDP information of a file, the server returns the information of multiple media streams contained in the media file to the client. Assign a trackID to each media stream. For example, video streams are allocated as track1 and audio streams are allocated as track2. After that, the client must specify in the URL to send the setup command for that track.

Therefore, we can think that mediasubsession represents a track of the media file on the server, that is, a media stream. Mediasession represents a media file on the server. For media files that contain both audio and video, mediasession contains two mediasubsessions.

Mediasession and mediasubsession only represent static information. If multiple clients request the same file, the server creates only one mediasession. Each client is public. A streamstate class is defined to differentiate the statuses of each mediasession, which is used to manage the statuses of each media stream. The Souce and sink connections are completed in mediasubsession. The Souce pointer is set to sink. When the sink needs data, it can be obtained by calling Souce's getnextframe.

 

Live555 uses a large number of simple factory modes, and each subclass has a createnew static member. The constructor of this subclass is set to protected. Therefore, it cannot be constructed externally by using new. At the same time, the constructor parameters of each class have a pointer to usageenvironment, so that you can output error messages and add yourself to scheduling.

 

4. hashtable

Live555 implements a simple hash table class basichashtable. In live555, this hash table class is used in many places. For example, the ing between media file names and mediasession, The cliing between sessionid and clientsession, And the usering between username and password.

 

5. SDP

SDP is the abbreviation of Session Description Protocol. It is an application layer protocol used to describe multimedia sessions. It is text-based and used to negotiate media types and encoding schemes during session creation. The client queries the media information of a specified file through the describe command. If you have any questions, you can refer to the previous article about RTSP, RTP, and RTCP.

 

6. Key class inheritance layers in live555 (the processing of h264 code streams is used as an example)

You can mix your face and familiarize yourself with it in detail later.

Souce

Hsf-videostreamframer is a real Souce used to read data from h264 files and assemble the data into frames. When the sink calls getnextframe, the frame data is returned to the sink.

 

Sink

Hsf-videortpsink is a real sink used to send frame data.

 

Subsession

Subsession is used to connect Souce and sink, and to manage each media stream.

For the h264 code stream, the flow direction of the data stream is:

Server: hsf-videostreamframer-> hsf-or5fragmenter (filter) r-> hsf-videortpsink

Client: hsf-rtpsouce-> sink (different implementations of different clients)

 

The relationship between the live555 categories is very complex. The relationship between the live555 categories increases the difficulty of learning the live555. Before learning more, you should first familiarize yourself with the basic process and understand the functions of each category, the details can be skipped for the moment.

For the live555 code style, I do not like it very much: first, the member variable naming method. Second, curly braces ({) follow the end of the previous line, without clear upper and lower alignment levels. Third, the multi-sentence code is located in the same line, and is mostly seen in the IF statement. Of course, this is just a personal liking. You are welcome to express your opinions.

At the end of the article, let's discuss how to pronounce live555. Many people have read [Liv] (Force v555 ). I personally feel wrong. Because live, as a verb, does read as force V, but it refers to residence, survival, and experience. As an adjective, It is live, live, and vivid. In this case, it should be read as: Lai v. As an open-source C ++ project that provides solutions for streaming media, it should be closer to live broadcasting! I personally think it should be read as lai v555. More foreign reading method: Lai v triple five! This is a practice that I take for granted. I have never heard of how foreigners read it. Thank you!

 

Next we will start with the server program and start to introduce the live555 source code.

2014.8.16 in Hangzhou, Zhejiang Province