Media streaming technology on the Apple platform HLS

Recently, chatting with friends about the media streaming server-side real-time transcoding technology, found that Apple's various terminal video playback did not use the common UDP-based RTSP/RTP, and forced the use of the HTTP Live stream technology, here a slightly summarized below.

Apple video streaming playback technical requirements

Apple's strong style has imposed technical restrictions on video data playback on its iphone, IPod, IPad, and Apple TV devices. First, the data compression format requirements for audio and video data are stated.

1. It is necessary to use the encoded compression format of H. IPhone 3G or newer, IPod touch second-generation or newer version can choose to support H. 3.1 Baseline, in order to be compatible with older versions of these devices, you need to support H. Baseline 3.0. For Ipad,apple TV 2 or later, you can choose to use Baseline Profile 3.0,baseline Profile 3.1 or main profile 3.1. The selection principle can be summed up as follows: Large screen video data using main Profile3.1, small screen video data using baseline Profile 3.1, in order to be compatible with the older version of the platform, need to support the baseline Profile version 3.0.

2. Audio encoding compression format

He-aac/aac-lc,stereo or MP3 (MPEG-1 Audio Layer 3), stereo.

Second, talk about the video streaming technology requirements

1. If the video stream is longer than 10 minutes or the video stream is over 5MB in five minutes, the HTTP Live stream technology scenario needs to be enforced.

2. If an app uses an HTTP Live stream in a wireless cellular network (such as a network such as 2G/3G), you need to provide at least one video stream at 64kbps or lower (for example, to provide a version that retains only sound and static images).

3. Software that does not meet the requirements will be removed from the rack.

It can be said that the Http Live Stream Technology (hereinafter referred to as HLS) is the core of Apple video playback scheme.

What is HLS

Wikipedia defines HLS as a fully HTTP-based media streaming protocol implemented by Apple. To understand HLS, first look at the typical network structure of HLS. In Apple's technical document description, the HLS network consists of three parts, the backend server side servers, the distribution module distribution, and the client clients.

The backend server generates the media data used for the final transfer, and the original voice video data (for example, the video and AAC audio) is packaged into a MPEG-2 transport stream (MPEG transport stream, as defined in the first part of the MPEG-2 specification, Also called MPEG-2 Transport Stream). MPEG-2 transmit data streams are segmented into pieces, for example, each piece is stored in a media file with a suffix of TS.

At the same time, generate an index file with a corresponding file suffix of m3u8.

The media stream index file and the corresponding series of media files are stored in the Distribution module distribution, the distribution module distribution is a standard Web server, through standard HTTP access, client clients can get to the media stream index file, and get the chunk TS file of the media stream one by one through the index file. Client clients can then stitch up the full video stream and display it to the final user. In contrast to the UDP-based RTSP media streaming playback scheme used on other mobile platforms, the distribution Module distribution only uses HTTP connections to transmit all the data, making its multimedia data flow the most extensive through the various edge network gateways, routers, network proxy devices. In addition, the distribution module distribution can work with a variety of front-end machines for digital content distribution, providing caching, distributed storage of media data, and more.

A typical example of an index file:

Here's a summary of the non-PC clients that support HLS, and all of Apple's devices, including the iphone, IPad, IPod, and Apple TV, are built-in to support HLS, and only support HLS, which does not support the RTSP media streams used by other platforms. The future version of Google's Android 3.0 also supports HLS.

Access the HLS video stream code in the Web page as follows:

The following is an example of the multimedia "real-time" transcoding software airplay video software on the Apple platform, and the packet capture analysis of HLS multimedia data stream.

"Real-time" transcoding of video data

Mobile Internet terminal only has limited multimedia codec ability, the reason analysis, one is limited by the platform's multimedia ability support, and secondly is limited by the copyright authorization, like PC computer, especially the domestic PC, almost unlimited on the system installed a variety of codecs, It has almost unlimited video codec capability. So, there is a kind of software on the Apple platform to complete the following functions, the PC computer installed a server, the client runs on the Ipad/iphone and other mobile internet terminals, the module running on the PC computer is responsible for the multimedia data stream "real-time" into

Mobile Internet terminal can support the code stream, Air Vedio/air Play It is such applications. This type of application is limited and the Apple platform only supports HLS, and it is bound to use HLS solutions only.

The following grab packets are caught on the server side of the air vedio on the PC side. The following HTTP request is the index file that gets the HLS multimedia stream.

The following HTTP request gets the media chunking file for HLS.