Research on the streaming media of iOS Ijkplayer framework based on B-station

Learning communication and technical discussion Sina Weibo attention: Geek James

One. Streaming media

Streaming media technology from the transmission form can be divided into: Progressive download and implementation of streaming media.

1. Progressive download

It is a way of playing between live play and local play, the progressive download does not have to wait until all the download is finished, you can download the side of the play, after the completion of the entire file will be saved. From the user's experience to the player's effect, progressive download and real-time streaming is no different, but progressive download keeps files on-premises. Here is a step-by-step download of the development of progressive download API and the local playback of the API is not much different, You can play with Mediaplayercontroller and Mediaplayerviewcontroller in the MediaPlayer frame, or you can use Avplayer in the avfoundation frame. In the Mac OS system there is an Apache HTTP server, first open the service, using the command line: sudo apachectl-v enter the password and then enter the sudo apachectl start can then put the file to play to/library/webs Erver/documents, you can do it.

2. Real-time Streaming media

Live streaming is playing on one side of the packet, without preserving a copy of the file locally, and the data is always delivered in real time. Users can fast forward and rewind, however, live streaming must ensure that the transmission speed of the packet is greater than the speed of the file playback, otherwise affect the playback effect.
The protocols for real-time streaming are: RTSP and HLS, MMS. HLS was proposed by Apple, which only requests basic HTTP messages, unlike RTSP and MMS, where HLS can pass through any firewall that allows HTTP data to pass through. Also, HLS has no special requirements for servers, as long as it can provide HTTP services.
HLS solution: First through the audio or video capture device to collect data, and then pass the data to the server to encode audio or video, encoding requirements in MPEG-2 format, the encoding is completed and then through the Media file separation tool to split, Then the partitioned files and their index files are published to the publisher, and then the client is accessible.

Two. HLS

HTTP Live Streaming (HLS) is Apple Inc. (Apple Inc) The implementation of the HTTP-based streaming media transmission protocol, can achieve streaming media live and on-demand, mainly in the iOS system, for iOS devices (such as the iphone, IPad) to provide audio and video live and on-demand programs. HLS on-demand, basically is the common segment HTTP on-demand, the difference is that its segmentation is very small. To achieve HLS on-demand, the focus is on the media file segmentation, there are many open source tools can be used, here I will no longer discuss, only talk about HLS live technology.

HLS implementation mode schematic diagram

Compared to the common streaming live broadcast protocol, such as RTMP protocol, RTSP protocol, MMS protocol, the biggest difference of HLS live is that the live client obtains, not a complete data stream. The HLS protocol stores the live stream as a continuous, short-length media file (mpeg-ts format) on the server side, while the client constantly downloads and plays these small files because the server side always generates new small files with the latest live data. So that the client as long as the sequential playback of the files obtained from the server, the implementation of the live broadcast. It can be seen, basically, that HLS is on-demand technical way to achieve live. Because the data through the HTTP protocol transmission, so completely do not consider the firewall or proxy problems, and the length of the fragmented file is very short, the client can quickly select and switch the bitrate to adapt to different bandwidth conditions of playback. However, this technical feature of HLS determines that its delay will always be higher than the normal streaming live protocol.

Based on the above understanding to implement the HTTP live streaming live, you need to study and implement the following technical key points

(1) Acquisition of video source and audio source data
(2) H264 encoding and AAC encoding of raw data
(3) Video and audio data encapsulated as Mpeg-ts package
(4) HLS segmentation Generation Strategy and m3u8 index file
(5) HTTP Transport Protocol

Three. B-station-based streaming media solution Ijkplayer Framework

Recently in the focus on streaming media live this piece, after the technology boss recommended B Station Ijkplayer Framework, began a crawl of the pit history ...
(1) The first step to GitHub is to download the B station Open source Ijkplayer framework:
GitHub website: B Station Open Source Ilkplayer
(2) According to the description of the document before compiling a variety of operations, in fact, it is not difficult to three major steps

Note: annotations with a # number callout are not entered in the console
The first step: the construction of the running environment (need ruby,home-brew,git,yams environment), open the terminal to enter the following statements without the # number.

# Install homebrew, git, yasm

Ruby-e "$ (curl-fssl https://raw.githubusercontent.com/Homebrew/install/master/install)"
Brew Install Git
Brew Install Yasm

# Add these lines to your ~/.bash_profile or ~/.profile# export android_sdk=# export android_ndk=# on Cygwin (unmaintained # install Git, make, yams

Step Two: clone the Ijkplayer project locally and integrate the FFmpeg into the Ijkplayer (this step is time-consuming and requires patience), enter the following command at the terminal.

git clone https://github.com/Bilibili/ijkplayer.git Ijkplayer-ios
CD Ijkplayer-ios
Git checkout-b Latest

./init-ios.sh

CD iOS
./compile-ffmpeg.sh Clean
./compile-ffmpeg.sh All

Step Three: open the Ijkmediodemo under the iOS file under Ijkplayer-ios
The following two sentences of the official mean to import Ijkmediaplayer into your project

#import Ios/ijkmediaplayer for Mediaplayer.framework-like Interface (recommended) #open ios/ijkmediademo/ Ijkmediademo.xcodeproj with Xcode

Then drag the ijkmediaplayer into the file.
This will open the official Demo sample program.

Run out of:


The above is the official demo, modeled after a lot of features are can be achieved. So the question is, how do you integrate Ijkplayer into your project? I met a big hole in the integration.

integrate Ijkplayer into your project:
To create your own project:
First step: ijkmediaframework.framework static libraries and ijkmediaplayer into the project file

Second: Open the project to add the two files to the home directory, and then open the build phases to add the following libraries

This is perfect, run the following can be easily and happily for the development of streaming media related ....

Four. m3u8

Do streaming media development must contact a file called m3u8 file, then what is m3u8?
The m3u8 file refers to the m3u file in the UTF-8 encoded format. The m3u file is an indexed plain text file that is opened when the playback software does not play it, but instead finds the corresponding audio and video file's network address for online playback based on its index.
M3U file tags and attribute descriptions
The m3u file can contain multiple tags, with each tag having the following functions and properties:

#EXTM3U

The first line of each m3u file must be this tag, please mark the function

#EXT-x-media-sequence:140651513

Each media URI has a unique ordinal number in playlist, adjacent to the ordinal +1, a media URI that does not have to be included, and if not, the default is 0

#EXTINF:,

Duration Specifies the duration (in seconds) of each media segment (TS), only valid for the URI that follows it, and title is the URL of the download resource

#EXT-x-targetduration

Specifies the maximum media segment length of time (in seconds). So the length of time specified in the #extinf must be less than or equal to the maximum value. This tag can only appear once in the entire playlist file (in the case of nesting, the tag will only appear for m3u8 with a real TS URL)

#EXT-x-key

Indicates how the media segments is decoded. Its scope is the next time the tag appears all the media URI, the property is None or AES-128. None indicates that the URI and IV (initialization Vector) attribute must not exist, AES-128 (Advanced Encryptionstandard) indicates that the URI must exist, and IV may not exist.
For AES-128, the Keytag and URI properties together represent a key file that can be obtained by URI, and if there is no IV (initialization Vector), the serial number is encoded as IV, The high-level of the serial number is assigned to 16 bytes of buffer, the left side is 0, and if there is an IV, the value is treated as a 16-byte 16 binary number.

#EXT-x-program-date-time

Associate an absolute time or date with the first sample in a media segment, valid only for the next Meida URI, in the form of #ext-x-program-date-time:
For example: #EXT-x-program-date-time:2010-02-19t14:54:23.031+08:00

#EXT-x-allow-cache

Whether to allow the cache, this can appear anywhere in the playlist file, and at most once, the effect is all media segments. The format is as follows: #EXT-x-allow-cache:

#EXT-x-playlist-type

Provides information about the variability of the PLAYLIST, which is valid for the entire PLAYLIST file, is optional, in the following format: #EXT-x-playlist-type:: If it is VOD, the server cannot change the PLAYLIST file; The server cannot change or delete any part of the playlist file, but it can add a new line to the file.

#EXT-x-endlist

Represents the end of the playlist, which can appear anywhere in the playlist, but only one, in the following format: #EXT-x-endlist

#EXT-x-media

A version of a language/translation that is used to represent the same content in a playlist, for example, by using 3 of these tags to represent 3 of audio without speech, or using 2 of this tag to represent different angles of video in playlists. This tag is independent, and the attribute contains:
URI: If not, the selectable version of the tag description exists in the Ext-x-stream-inf of the main playlist;
Type:audio and VIDEO;
Group-id: Mediatag with the same ID, forming a set of styles;
LANGUAGE: Determining the primary language used
NAME: Translation of human-readable languages
Default:yes or no, the default is no, and if yes, the client will play this option unless the user chooses to do so.
Autoselect:yes or no, the default is no, and if yes, the client chooses according to the current playback environment (the user does not choose according to his preference).

#EXT-x-stream-inf

Specifies a media URI that contains multimedia information as playlist, which is generally used as a nested m3u8, which is valid only for the URI immediately following it, in the following format: #EXT-x-stream-inf: There are some properties:
BANDWIDTH: bandwidth, must have.
Program-id: This value is a decimal integer that uniquely identifies a specific description within the scope of the playlist file. A playlist file may contain multiple of this tag with the same ID.
CODECS: Not required.
RESOLUTION: Resolution.
AUDIO: This value must match the value of the "Group-id" attribute in the "Ext-x-media" tab of the audio category.
VIDEO: Ditto

Research on the streaming media of iOS Ijkplayer framework based on B-station