Adaptive streaming one of the learning notes

Have time to do something. I'm here to record some of the things I've learned about multimedia information systems, and I hope to be able to benefit from my career in the future. Starting today, I'm going to show you how to master the series of learning notes from the mainstream adaptive streaming technology from scratch.

Let's start with a few useful links:

http://www.zhihu.com/question/20621558

Http://www.garymcgath.com/streamingprotocols.html

Http://www.streamingmedia.com/Articles/Editorial/What-Is-.../what-is-a-streaming-media-protocol-84496.aspx

Http://www.streamingmedia.com/Articles/Editorial/What-Is-.../what-is-adaptive-streaming-75195.aspx

The above link to the mainstream streaming from the various periods of the agreement is briefly introduced and the general comparison, can give you some more core understanding. Streaming technology from the origin of development is divided into two major schools: based on the network protocol and need the corresponding multimedia server support RTSP, Microsoft's MMS and Adobe rtmp; HTTP-based streaming scenarios such as Apple HLS, Microsoft MSS, Adobe's new HDS and MPEG DASH as an international standard. Although network protocol-based scenarios such as rtmp are still receiving widespread use and support, there is no doubt that HTTP-based steaming solutions are the future trend because of their unique advantages. Of course, it is also noted that peer streaming using BitTorrent technology has its advantages on some occasions.

And our work is hoped that above the theoretical knowledge above, to the existing mainstream video website's streaming technology carries on the actual research.

1. Youtube

On-demand: More special, using the HTTPS protocol for transmission, so at most only through browser debugging to get its GET request header information. YouTube uses an HTTP playback method based on the HTML5 player, with a dozens of-megabyte segment for each video, and then a GET request for a small shard each time, the fragment size of each request is roughly the magnitude of 1mb+.

HTTPS://R3---sn-oguesnze.googlevideo.com/videoplayback?clen=29595934&gir=yes&keepalive=yes&mime= Video%2fmp4&key=yt6&itag=135&nh=igpwcjazlm5ydde5kgkxmjcumc4wlje&lmt=1431925619201789&sver= 3&initcwndbps=20271250&sparams=clen%2cdur%2cgir%2cid%2cinitcwndbps%2cip%2cipbits%2citag%2ckeepalive% 2clmt%2cmime%2cmm%2cmn%2cms%2cmv%2cnh%2cpl%2crequiressl%2csource%2cupn%2cexpire&ipbits=0&expire= 1452461073&mm=31&pl=25&mn=sn-oguesnze&id=o-acr7mvfk8xbley2goctmhrq6kq52qvzb1aqscmla7qcw& dur=271.070&requiressl=yes&ip=45.32.14.175&mt=1452439318&mv=m&ms=au&fexp=9407155% 2c9408503%2c9408940%2c9412859%2c9416126%2c9418401%2c9419444%2c9420452%2c9420718%2c9421906%2c9422341%2c9422596% 2c9423662%2c9424629%2c9424753%2c9426047%2c9426718&source=youtube&signature= a26ea9b83133d04e506fba26cfdc7801ca5f7376.67426df786258d3aa7bf6c56e34f87282e35648e&upn=2ni73vtpabe&cpn= Pgwstojvits1oreq&alr=yes&ratebypass=yes&c=web&cver=html5& range=4273674-6235380&rn=30&rbuf=32999

HTTPS://R3---sn-oguesnze.googlevideo.com/videoplayback?clen=29595934&gir=yes&keepalive=yes&mime= Video%2fmp4&key=yt6&itag=135&nh=igpwcjazlm5ydde5kgkxmjcumc4wlje&lmt=1431925619201789&sver= 3&initcwndbps=20271250&sparams=clen%2cdur%2cgir%2cid%2cinitcwndbps%2cip%2cipbits%2citag%2ckeepalive% 2clmt%2cmime%2cmm%2cmn%2cms%2cmv%2cnh%2cpl%2crequiressl%2csource%2cupn%2cexpire&ipbits=0&expire= 1452461073&mm=31&pl=25&mn=sn-oguesnze&id=o-acr7mvfk8xbley2goctmhrq6kq52qvzb1aqscmla7qcw& dur=271.070&requiressl=yes&ip=45.32.14.175&mt=1452439318&mv=m&ms=au&fexp=9407155% 2c9408503%2c9408940%2c9412859%2c9416126%2c9418401%2c9419444%2c9420452%2c9420718%2c9421906%2c9422341%2c9422596% 2c9423662%2c9424629%2c9424753%2c9426047%2c9426718&source=youtube&signature= a26ea9b83133d04e506fba26cfdc7801ca5f7376.67426df786258d3aa7bf6c56e34f87282e35648e&upn=2ni73vtpabe&cpn= Pgwstojvits1oreq&alr=yes&ratebypass=yes&c=web&cver=html5& range=6235381-7711377&rn=32&rbuf=43270

HTTPS://R3---sn-oguesnze.googlevideo.com/videoplayback?clen=29595934&gir=yes&keepalive=yes&mime= Video%2fmp4&key=yt6&itag=135&nh=igpwcjazlm5ydde5kgkxmjcumc4wlje&lmt=1431925619201789&sver= 3&initcwndbps=20271250&sparams=clen%2cdur%2cgir%2cid%2cinitcwndbps%2cip%2cipbits%2citag%2ckeepalive% 2clmt%2cmime%2cmm%2cmn%2cms%2cmv%2cnh%2cpl%2crequiressl%2csource%2cupn%2cexpire&ipbits=0&expire= 1452461073&mm=31&pl=25&mn=sn-oguesnze&id=o-acr7mvfk8xbley2goctmhrq6kq52qvzb1aqscmla7qcw& dur=271.070&requiressl=yes&ip=45.32.14.175&mt=1452439318&mv=m&ms=au&fexp=9407155% 2c9408503%2c9408940%2c9412859%2c9416126%2c9418401%2c9419444%2c9420452%2c9420718%2c9421906%2c9422341%2c9422596% 2c9423662%2c9424629%2c9424753%2c9426047%2c9426718&source=youtube&signature= a26ea9b83133d04e506fba26cfdc7801ca5f7376.67426df786258d3aa7bf6c56e34f87282e35648e&upn=2ni73vtpabe&cpn= Pgwstojvits1oreq&alr=yes&ratebypass=yes&c=web&cver=html5& range=7711378-9099835&rn=33&rbuf=49449

Live: Using the DASH protocol, the typical way is to divide the live video stream and audio stream into 1mb+ and dozens of kb+ magnitude. Get the manifest file by first GET request to get play_list, and then according to the current network status or user needs real-time select corresponding bitrate video and audio fragments.

https://manifest.googlevideo.com/api/manifest/dash/itag/0/hfr/1/pmbypass/yes /fexp/9407155%2c9408503%2c9408940%2c9412859%2c9416126%2c9418401%2c9419444%2c9420452%2c9420718%2c9421906% 2c9422341%2c9422596%2c9423662%2c9424629%2c9424753%2c9426047%2c9426718/sver/3/source/yt_live_broadcast/upn/ 98vtqug-5_g/gcr/jp/ipbits/0/as/fmp4_audio_clear%2cfmp4_sd_hd_clear/sparams/gcr%2chfr%2cid%2cip%2cipbits%2citag %2cplaylist_type%2cpmbypass%2csource%2cexpire/signature/ 4d3a6ce9b09129f5feaf1b3f51cbd94042cdab5e.6e333d90654565e7dc9b106d93a54a846d7d5828/playlist_type/dvr/expire/ 1452462185/key/yt6/ip/45.32.14.175/id/y60wdzzt8yg.1?cpn=awum5pvwpt0gfj7r&mpd_version=3&start_seq= 2634917

HTTPS://R9---sn-oguesn7s.googlevideo.com/videoplayback?id=y60wdzzt8yg.1&itag=136&source=yt_live_ Broadcast&requiressl=yes&pmbypass=yes&gcr=jp&playlist_type=dvr&ratebypass=yes&cmbypass =yes&mime=video%2fmp4&live=1&gir=yes&fexp= 9407155,9408503,9408940,9412859,9416126,9418401,9419444,9420452,9420718,9421906,9422341,9422596,9423662,9424629,9424753,9 426047,9426718&sver=3&upn=98vtqug-5_g&signature= 2693eea04dcfcaa687547b92275c4d5b5a84c4e4.5c12f9a2c66f4d5f715944bd0a30ae38f30dc5e8&key=cms1&cpn= Awum5pvwpt0gfj7r&mpd_version=3&ip=45.32.14.175&ipbits=0&expire=1452462185&sparams=cmbypass , Expire,gcr,gir,id,ip,ipbits,itag,live,mime,mm,mn,ms,mv,nh,pl,playlist_type,pmbypass,ratebypass,requiressl, Source&mm=32&mn=sn-oguesn7s&ms=lv&mt=1452440510&mv=u&nh= igpwcjazlm5ydde5kgkxmjcumc4wlje&pl=25&alr=yes&keepalive=yes&c=web&cver=html5&sq= 2634917&clen=1081219&lmt=1452440650952458&dur=5.0&rn=144&rbuf=9492

HTTPS://R9---sn-oguesn7s.googlevideo.com/videoplayback?id=y60wdzzt8yg.1&itag=140&source=yt_live_ Broadcast&requiressl=yes&pmbypass=yes&gcr=jp&playlist_type=dvr&ratebypass=yes&cmbypass =yes&mime=audio%2fmp4&live=1&gir=yes&fexp= 9407155,9408503,9408940,9412859,9416126,9418401,9419444,9420452,9420718,9421906,9422341,9422596,9423662,9424629,9424753,9 426047,9426718&sver=3&upn=98vtqug-5_g&signature= 76e310685015414b7b9859dd3e963e1b21b8250e.786b6c6e1a779e932fd971116fa25c3157b8afd8&key=cms1&cpn= Awum5pvwpt0gfj7r&mpd_version=3&ip=45.32.14.175&ipbits=0&expire=1452462185&sparams=cmbypass , Expire,gcr,gir,id,ip,ipbits,itag,live,mime,mm,mn,ms,mv,nh,pl,playlist_type,pmbypass,ratebypass,requiressl, Source&mm=32&mn=sn-oguesn7s&ms=lv&mt=1452440510&mv=u&nh= igpwcjazlm5ydde5kgkxmjcumc4wlje&pl=25&alr=yes&keepalive=yes&c=web&cver=html5&sq= 2634917&clen=79580&lmt=1452440650952458&dur=5.0&rn=145&rbuf=9492

2, Youku, Iqiyi and other domestic mainstream video-on-demand website

On-demand: A typical way is to divide a large video into large fragments at a level of dozens of MB. Then through the HTTP protocol, simply give the desired video via GET request. The length range of the flv file, unlike YouTube, is that the size of each request is 10mb+ or so, while the file is downloaded and played until the file is played. The following is a simple request process:

http://112.29.146.4/videos/v0/20160110/ea/ee/ 57ae4676c08dc2540b6d8c774dc4b838.f4v?key=01bfa2a0b5d4d0d2debcb7e0f384ac3b1&src=iqiyi.com&qd_tvid= 439500800&qd_vipres=0&qd_index=4&qd_aid=439500800&qd_stert=1080970&qd_scc= 3040a4319737bb34a3f031ec53cb09f7&qd_sc=ba9bd8d9c15916c0dd71deeda26dfddf&qd_src= 1702633101b340d8917a69cf8a4b8c7c&qd_ip=6f0a207b&qd_uid=0&qd_tm=1452425434000&qd_vip=0&ran= 10900335&qyid=c828a7e5701c3bbabd54b174591a13c1&qypid=439500800_11&s=144756.3774431001&bt= 175184&z=hefei_cmnet&la=cmnet| Chongqing-111.10.35.152&li=hefei_cmnet&lsp=141&lc=83&uuid=6f0a2398-56926b71-6f& range=0-10747903 &qypid=439500800_01010011010000000000_1&ran=10900604

http://112.29.146.4/videos/v0/20160110/ea/ee/ 57ae4676c08dc2540b6d8c774dc4b838.f4v?key=01bfa2a0b5d4d0d2debcb7e0f384ac3b1&src=iqiyi.com&qd_tvid= 439500800&qd_vipres=0&qd_index=4&qd_aid=439500800&qd_stert=1080970&qd_scc= 3040a4319737bb34a3f031ec53cb09f7&qd_sc=ba9bd8d9c15916c0dd71deeda26dfddf&qd_src= 1702633101b340d8917a69cf8a4b8c7c&qd_ip=6f0a207b&qd_uid=0&qd_tm=1452425434000&qd_vip=0&ran= 10900335&qyid=c828a7e5701c3bbabd54b174591a13c1&qypid=439500800_11&s=144756.3774431001&bt= 175184&z=hefei_cmnet&la=cmnet| Chongqing-111.10.35.152&li=hefei_cmnet&lsp=141&lc=83&uuid=6f0a2398-56926b71-6f& range=10747904-24379391 &qypid=439500800_01010011010000000000_1&ran=10931290

http://112.29.146.4/videos/v0/20160110/ea/ee/57ae4676c08dc2540b6d8c774dc4b838.f4v?key= 01bfa2a0b5d4d0d2debcb7e0f384ac3b1&src=iqiyi.com&qd_tvid=439500800&qd_vipres=0&qd_index=4& Qd_aid=439500800&qd_stert=1080970&qd_scc=3040a4319737bb34a3f031ec53cb09f7&qd_sc= Ba9bd8d9c15916c0dd71deeda26dfddf&qd_src=1702633101b340d8917a69cf8a4b8c7c&qd_ip=6f0a207b&qd_uid=0 &qd_tm=1452425434000&qd_vip=0&ran=10900335&qyid=c828a7e5701c3bbabd54b174591a13c1&qypid= 439500800_11&s=144756.3774431001&bt=175184&z=hefei_cmnet&la=cmnet| chongqing-111.10.35.152&li=hefei_cmnet&lsp=141&lc=83&uuid=6f0a2398-56926b71-6f&range= 24379392-38239153&qypid=439500800_01010011010000000000_1&ran=10998356

In the example above, the file fragment is 38,239,153 bytes long and is divided into three get requests to complete the download. Client-based flash technology, can be connected to the received portions of the file for real-time decoding implementation side play side download, which is provided by the technology of Adobe Guarantee.

3, PPTV, LeTV, live show and other domestic video broadcast site

PPTV-On-demand: As we all know, PPTV is born out of PPLive, so its technology based on peer stream is different from other mainstream video-on-demand, which is also reflected in its on-demand business. We found that on-demand video on the PPTV website is usually divided into dozens of MB fragments, and then the HTTP request takes about 1mb+ shards at a time. Each acquisition process differs from other sites. Initialization is a link (the returned information does not actually have a specific data content, just the basic information about the fragment):

http://122.72.99.58/1/0/1023/ece14c23db607048bfa27f8e682f00e1.mp4?fpp.ver=1.3.0.19&type=web.fpp&k= 292f456fcf0c2ba9fe41ad897bfe0dba-6edb-1452498293&get_drm_header=true

Then through the dozens of GET request, each time get about 1MB more shards, until the entire fragment has been downloaded.

、

· · · · · · · · · · · · · · · · · · · · · · · ·

The format of the above get request is as follows:

http://122.72.99.58/1/17040384/0/ece14c23db607048bfa27f8e682f00e1.mp4?fpp.ver=1.3.0.19&key= Df08b453543a03d9edcaa4c45508754c&k=292f456fcf0c2ba9fe41ad897bfe0dba-6edb-1452498293&type=web.fpp

You can see that there is a difference between the link format and the GET request that is just to get the video clip information. It is worth mentioning that the offset address in the PPTV scheme is defined by a number that is divided by three slashes before the video file name in the link: ***/1/17040384/0/***. "1" indicates the sequence number of the fragment, "17040384" indicates the starting offset address in the fragment, and "0" indicates until the end. By grasping the packet results, it is found that the response of the request server is not necessarily to the end, but the transfer of about 1MB will stop, and then start the next similar get request.

PPTV Live: The way to live is relatively simple, directly after the live stream encoded into block (I guess this is possible TS stream?). It can be further explored and indexed, and each GET request receives 5 blocks of content in HTTP mode, roughly hundreds of KB in magnitude.

4, the traditional monitoring video probe

Basically, it is done through the traditional RTSP protocol. All you need to do is search the Internet for several available RTSP streams to test with VLC to discover.

5, iOS and Android phone client testing

On-demand testing under the iOS system: (Take Iqiyi as an example)

"Mobile Bag Capture method: Http://www.chinaz.com/web/2015/0326/393344.shtml"

This problem is divided into two aspects, due to the particularity of the iOS system, is divided into the app side and Safari browser side. Due to the high controllability of the app-side manufacturers, we find that the traditional FLV container format is still adopted by grasping the packet, and it should be encoded and decoded in the app by a third-party open source codec library, such as FFmpeg. For Safari browsers, because iOS only supports HTML5, flash controls are not available at all, so vendors tend to use apple-led HLS solutions. The crawled HTTP request link is as follows:

Http://data.video.iqiyi.com/videos/v0/20160112/91/dd/19ad0a1686a4f7d7cc5c41e666bac06f.ts?qypid=440440000_31 &start=2674193&end=5408962&hsize=76694&tag=1&v=&contentlength=1847852&qd_uid=0 &qd_vip=0&qd_src=d846d0c32d664d32b6b54ea48997a589&qd_tm=1452616355776&qd_ip=111.10.33.180& Qd_sc=74d976bf20c2a2dbfb15791d126c2a04

Http://data.video.iqiyi.com/videos/v0/20160112/91/dd/19ad0a1686a4f7d7cc5c41e666bac06f.ts?qypid=440440000_31 &start=4785056&end=6064737&hsize=76694&tag=1&v=&contentlength=868184&qd_uid=0& Qd_vip=0&qd_src=d846d0c32d664d32b6b54ea48997a589&qd_tm=1452616355776&qd_ip=111.10.33.180&qd_sc =74d976bf20c2a2dbfb15791d126c2a04

Http://data.video.iqiyi.com/videos/v0/20160112/91/dd/19ad0a1686a4f7d7cc5c41e666bac06f.ts?qypid=440440000_31 &start=5708731&end=6892905&hsize=76694&tag=1&v=&contentlength=826448&qd_uid=0& Qd_vip=0&qd_src=d846d0c32d664d32b6b54ea48997a589&qd_tm=1452616355776&qd_ip=111.10.33.180&qd_sc =74d976bf20c2a2dbfb15791d126c2a04

As you can see, the requested video file in the link is a TS stream, the TS stream file should be a large file, combined with a link given begin and end of the requested video fragment start address, at the same time, the length of the fragment is given, and the other slightly overlapping TS stream fragments, So that it can be played smoothly. Using TS streaming files as a standard container is one of the main features of Apple-led HLS streaming Media transport protocol.

And for the live, can also be solved with similar ideas, because HLS is native to support video streaming protocol. This is also reflected in apps on other iOS devices, such as CNTV. In the capture monitoring of its live video, we not only found the TS file stream, but also found more details, such as the process of getting the m3u8 file. This further illustrates the unity of the HLS protocol on iOS devices.

"m3u8 file parsing: http://blog.sina.com.cn/s/blog_6cf7acdf0102v0xv.html"

Adaptive Streaming One of the learning notes