IPhone HTTP streaming with FFMPEG and an open source segmenter

 

IPhone HTTP streaming with FFMPEG and an open source segmenterby Carson |
Published: June 28,200 9

With the release of the iPhone OS 3 Update came the ability to do live streaming. there are a few types of streaming and each requires a certain encoding and segmentation. I 've put together a cheat sheet on how I went about building a static stream using
FFMPEG and an example segmenter that someone has posted. i'm not covering into wed streams in this post but if you are thinking about implementing a random wed stream the following will help you make a step in that direction.

Before getting started it is best to read over the apple documentation on HTTP live streaming. start out with

IPhone streaming media overview. This document covers the basics of how the streaming works and has some nice diagrams.

If you want even more information after reading the overview you can take a look at
HTTP live streaming draft proposal that was submitted to the IETF by Apple. It covers the Streaming Protocol in complete detail and has examples of the stream file format for reference.

Once you are ready to start grab a decent quality video clip to use. If you don't have one handy I found a nice list of downloadable

HD clips in various formats for testing.

Step 1: Grab the latest version of FFMPEG

You may be able to get away with anything after FFMPEG 0.5 but you might as well pull down a more recent version. The FFMPEG
Download Page has instructions on getting the latest version. I pulled the version I used out of git.

I used the following command to configure FFMPEG:

configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libx264

One of the main things to note is the -- enable-libx264 flag.

Step 2: encode your video for the iPhone

Once you have a working version of FFMPEG it is time to create an x264 encoded stream that will work with the iPhone. There are a few things to note before diving in:

1. The supported bitrates for streaming are: 100 Kbps to 1.6 Mbps
2. The suggested bitrates for streaming are *:
   • Low-96 kbps video, 64 Kbps audio
   • Medium-256 kbps video, 64 Kbps audio
   • High-800 kbps video, 64 Kbps audio
3. The iPhone screen size is: 480x320

* See Step 7 for more information on what I think are better bitrates.

Taking all that into account someone on the iPhone developer forums suggested the following and it works well for me:

FFmpeg-I <in File>-F mpegts-acodec libmp 3lame-ar 48000-AB 64 K-s 320 × 240-vcodec libx264-B 96 k-flags + loop-CMP + chroma -partitions + parti4x4 + partp8x8 + partb8x8-subq 5-trellis 1-Refs 1-coder
0-me_range 16-keyint_min 25-SC _threshold 40-I _qfactor 0.71-BT 200 k-maxrate 96 k-bufsize 96 k-rc_eq 'blurcplx ^ (1-qcomp) '-qcomp 0.6-qmin 10-Qmax 51-qdiff 4-level 30-aspect 320: 240-G 30-async 2 <output file>

If you want some more detail on some of these commands check out
X264 Encoding Guide and in general
FFmpeg documentation to see what all the flags mean.

Note that I have the bitrate set to 96 k in the above example. that can be changed to fit your needs. use the script that I have created later in the post or just make sure you change the-B,-maxrate, and-bufsize values.

Step 3: Download and build the segmenter

Now you have a complete video but you don't want to toss the entire thing up or you wouldn't be reading about HTTP streaming. what you need is a way to segment the video stream into smaller chunks. you can download Apple's segmenter (See the overview above)
For more information on where to find it) or you can download one created by the forum user
Corp186.

There is an SVN repository set up for
Segmenter source. it is only a couple files and it is easy to build. the trouble you may run into is that the makefile that it comes with won't build the binary correctly. don't worry it just takes some extra link flags to make it work. the following is
What I needed in the makefile to get it to build on my system:

ALL:
Gcc-wall-G segmenter. C-o segmenter-lavformat-lavcodec-lavutil-lbz2-lm-LZ-lfaac-lmp3lame-lx264-lfaad

Clean:
Rm segmenter

After compiling the segmenter you are ready to create your first HTTP streaming content.

The format of the segmenter command is:

Segmenter <input MPEG-TS File> <segment duration in seconds> <output MPEG-TS file prefix> <output m3u8 Index File> Following is an example used to create a stream from a video file created with the above FFMPEG command split into 10 second intervals:

Segmenter sample_low.ts 10 sample_low stream_low.m3u8 http://www.ioncannon.net/Step 4: Prepare the HTTP server

At this point you shoshould have a set of files that represent the stream and a stream definition file. those files can be uploaded to a Web server at this point but there is another important step to take that ensures they will be downloaded correctly and that
Is setting up MIME types. There are two MIME types that are important for the streaming content:

. M3u8 application/X-mpegurl
. Ts video/mp2t

If you are using Apache you wowould want to add the following to your httpd. conf file:

Addtype application/X-mpegurl. m3u8
Addtype video/mp2t. Ts

If you are using Lighttpd you wowould want to put this in your configuration file (if you have other MIME types defined make sure you just add these and don't set them ):

Mimetype. Assign = (". m3u8" => "application/X-mpegurl", ". Ts" => "Video/mp2t") Step 5: test the stream

The video is encoded for the iPhone, segmented for streaming, and the server is configured. the only thing left to do is test the stream and the fastest way to do that is to use the new HTML5 video tag. here is an example of how to set it up:

<HTML>
<Head>
<Title> video test </title>
<Meta
Name = "viewport"
Content = "width = 320; initial-scale = 1.0; maximum-scale = 1.0; User-scalable = 0;"/>
</Head>
<Body
Style = "background-color: # ffffff;">
<Center>
<Video width = '000000'
Height = '20180101'
Src = "stream-128k.m3u8"
/>
</Center>
</Body>
</Html>

If everything has been done correctly you shoshould see the video.

If you want to test the stream out in an application then
Download the movieplayer iPhone demo application from the iPhone developer site. build and run it in the simulator or put it on an actual phone and then type the URL in for the server you uploaded your stream.

That is all there is to building a single static HTTP stream. A good number of steps but if you have some experience using FFMPEG It isn' t too hard to set up. the only pitfalls I ran into revolve around und trying to segment the stream without the segmeter code.
I don't know enough about how the segmentation works to know why this is so difficult to do but I believe it cocould have something to do with synchronization points in the stream. of course when you stray from the path the stream just doesn't work and you get
A generic error message so that is just my best guess. I'll also guess that Apple may tighten up the player over time and make it work better with Miss-formatted streams.

Step 6: automating the stream encoding and Segmentation

Here is a little script I put together that first encodes an input file and then segments it into 10 second chunks:

#! /Bin/sh

BR = 800 K

FFmpeg-I $1
-F mpegts-acodec libmp 3lame-ar 48000
-AB 64 K-s 320 × 240
-Vcodec libx264-B $ br
-Flags + loop-CMP + chroma-partitions + parti4x4 + partp8x8 + partb8x8
-Subq 5-trellis 1
-Refs 1-coder 0-me_range 16-keyint_min 25-SC _threshold 40-I _qfactor 0.71
-BT 200 k-maxrate
$ Br-bufsize $ Br-rc_eq
'Blurcplx ^ (1-qcomp) '-qcomp 0.6
-Qmin 10-Qmax 51-qdiff 4
-Level 30-aspect 320: 240
-G 30-async 2 sample _ $ br_pre.ts

Segmenter sample _ $ br_pre.ts 10 sample _ $ BR stream-$ Br. m3u8 http://www.ioncannon.net/

Rm-F sample _ $ br_pre.ts

The script cocould use some work but it does a good enough job for testing.

Step 7: Create a variable rate HTTP stream

Once you have creating a single stream down you need to try out creating a variable bitrate stream. there isn't much to it, just create different bitrate encoded streams and link to their stream definition files in a separate stream definition file. here
Is an example:

# Extm3u
# EXT-X-STREAM-INF: Program-id = 1, bandwidth = 96000

Http: // 192.168.132.15/IPV/stream-96k.m3u8

# EXT-X-STREAM-INF: Program-id = 1, bandwidth = 256000

Http: // 192.168.132.15/IPV/stream-256k.m3u8

# EXT-X-STREAM-INF: Program-id = 1, bandwidth = 800000

Http: // 192.168.132.15/IPV/stream-800k.m3u8

I gave the above a try using both the iPhone's 3G connection and a WiFi connection. The following log shows the two different attempts (first 3G then WiFi ):

6.x.x.x ioncannon.net - [20:49:13] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."6.x.x.x ioncannon.net - [20:49:14] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."6.x.x.x ioncannon.net - [20:49:15] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."6.x.x.x ioncannon.net - [20:49:16] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."6.x.x.x ioncannon.net - [20:49:18] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."6.x.x.x ioncannon.net - [20:49:21] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."6.x.x.x ioncannon.net - [20:49:30] "GET /s_96k-00003.ts HTTP/1.1" 200 383520 "-" "..."6.x.x.x ioncannon.net - [20:49:32] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."6.x.x.x ioncannon.net - [20:49:39] "GET /s_256k-00003.ts HTTP/1.1" 200 716844 "-" "..."6.x.x.x ioncannon.net - [20:49:49] "GET /s_256k-00004.ts HTTP/1.1" 200 705564 "-" "..."6.x.x.x ioncannon.net - [20:49:57] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."6.x.x.x ioncannon.net - [20:49:59] "GET /s_96k-00004.ts HTTP/1.1" 200 368668 "-" "..."6.x.x.x ioncannon.net - [20:50:03] "GET /s_96k-00005.ts HTTP/1.1" 200 371300 "-" "..."6.x.x.x ioncannon.net - [20:50:13] "GET /s_96k-00006.ts HTTP/1.1" 200 398936 "-" "..."6.x.x.x ioncannon.net - [20:50:16] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."6.x.x.x ioncannon.net - [20:50:22] "GET /s_256k-00006.ts HTTP/1.1" 200 758016 "-" "..."6.x.x.x ioncannon.net - [20:50:36] "GET /s_256k-00007.ts HTTP/1.1" 200 737524 "-" "..."6.x.x.x ioncannon.net - [20:50:40] "GET /s_256k-00008.ts HTTP/1.1" 200 773244 "-" "..."6.x.x.x ioncannon.net - [20:50:46] "GET /s_256k-00009.ts HTTP/1.1" 200 717032 "-" "..."6.x.x.x ioncannon.net - [20:50:57] "GET /s_256k-00010.ts HTTP/1.1" 200 768920 "-" "..."6.x.x.x ioncannon.net - [20:51:06] "GET /s_256k-00011.ts HTTP/1.1" 200 611000 "-" "..."1.x.x.x ioncannon.net - [20:52:23] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."1.x.x.x ioncannon.net - [20:52:24] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."1.x.x.x ioncannon.net - [20:52:25] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."1.x.x.x ioncannon.net - [20:52:25] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."1.x.x.x ioncannon.net - [20:52:26] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."1.x.x.x ioncannon.net - [20:52:27] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."1.x.x.x ioncannon.net - [20:52:28] "GET /stream-800k.m3u8 HTTP/1.1" 200 730 "-" "..."1.x.x.x ioncannon.net - [20:52:31] "GET /s_800k-00002.ts HTTP/1.1" 200 1774156 "-" "..."1.x.x.x ioncannon.net - [20:52:34] "GET /s_800k-00003.ts HTTP/1.1" 200 1916096 "-" "..."1.x.x.x ioncannon.net - [20:52:38] "GET /s_800k-00004.ts HTTP/1.1" 200 1831872 "-" "..."1.x.x.x ioncannon.net - [20:52:41] "GET /s_800k-00005.ts HTTP/1.1" 200 1831496 "-" "..."1.x.x.x ioncannon.net - [20:52:46] "GET /s_800k-00006.ts HTTP/1.1" 200 1967608 "-" "..."1.x.x.x ioncannon.net - [20:52:50] "GET /s_800k-00007.ts HTTP/1.1" 200 1676208 "-" "..."1.x.x.x ioncannon.net - [20:52:54] "GET /s_800k-00008.ts HTTP/1.1" 200 2094132 "-" "..."1.x.x.x ioncannon.net - [20:52:58] "GET /s_800k-00009.ts HTTP/1.1" 200 1860260 "-" "..."1.x.x.x ioncannon.net - [20:53:08] "GET /s_800k-00010.ts HTTP/1.1" 200 2008404 "-" "..."1.x.x.x ioncannon.net - [20:53:19] "GET /s_800k-00011.ts HTTP/1.1" 200 1400224 "-" "..."

Notice that there is a decent bit of indecisiveness on the part of what stream to pick when using 3G. for my test it actually caused the player to pause while it switched from the 256 k stream back to the 96 k stream. the stream on the WiFi connection starts
Out low but then jumps right to the highest quality and stays there. overall it seems like the variable rate streaming works decently and again Apple may be able to tweak it down the road to get even better results.

The bitrate jump between 96 k and 256 k is probably too large even though that is what Apple seems to recommend. I believe with some testing a better set of bitrates cocould be found. the video quality of the 256 k bitrate looks pretty good so I wocould say that
96 k, 128 K and 384 K wocould potentially be a better choice.

Some parting notes

If you are interested in how the segmenter works you can find out more on how to use libavformat at the following resources:
An older libavformat tutorial,

Some sample libavformat code, how to write a video player in less than 1000 lines, and

More sample libavformat code.

The next step for this is to do a previous wed live stream. I 've done a little experimenting so far and with a modified segmeter I can generate a live stream. I will need to heavily modify the segmeter to get a live broadcast wed stream so it may take a little while
To get it done. My intent of course will be to combine the modifications with something fun like S3 and cloudfront since I believe that wocould be a sweat combination.