IOS Video Live/Smart Home (line of code, zero-based) lesson:1 overall architecture

Tag: The target response CLI parameter of the net IDT scribe file gravity

Some time ago due to work need to do a video live/smart home class application. A summary of streaming media processing for iOS audio and video columns. Here want to record the entire development process, on the one hand, and everyone to explore the study, on the other hand can also be convenient to check the leak after the vacancy.

The entire development does not rely on any third-party framework, all streaming protocols are a row of lines knocked Up, why? The fish is better than the award

Let's start by understanding the architecture of the entire software.

Front end we have introduced in the iOS audio&video column very detailed, including the camera, the audio and video processing all has done the detailed analysis. Back end We also made a detailed introduction to the iOS audio&video column in the FFmpeg, advanced use, also no longer discussed here. The following is a detailed analysis of the network section.

(1) RTCP

RTCP:RTP control Protocol, real-time transmission protocol, general and RTP use, mainly for data transmission monitoring, control functions. At the same time RTCP is based on UDP transmission. This information transmitted by RTCP is important, including: timestamps (for synchronization), serial numbers (for packet drops and reorder detection), and payload formats (encoding formats for data). A simple summary of this is for QoS feedback and synchronous media streaming. In comparison with RTP, half of its bandwidth is about 5% of RTP. Very small.

Depending on the state of use, RTCP is divided into the following situations:

Live, smart home projects just use the Send side report, here we will in the following code in detail analysis.

(2) RTP

Real-time Transport Protocol, real-time transmission protocol, generally used for the transmission of multimedia data. Audio and video mainly through this protocol transmission, it is built on the UDP protocol, more efficient but allow packet loss. So in the media reorganization need to do a lot of work. The RTP protocol package format is as follows:

(3) SDP

Let's take a look at the packet-grabbing network request process:

Server response data. Part of our concern: RTSP

response=rtsp/1.0 OK

Cseq:1

content-base:rtsp://192.168.36.168/

date:2015 July 29 Gmt+8 2:52:00

Content-type:application/sdp

content-length:424

SDP section:

V=0

o=-1804289383 1804289383 in IP4 192.168.36.168

S=livestream from IOS

C=INIP4 0.0.0.0

t=00

a=control:*

M=video0 RTP/AVP 96

b=tias:85528

a=maxprate:9.0000

A=control:streamid=1

a=rtpmap:96h264/90000

a=mimetype:string; " video/h264 "

a=framesize:96720-480

a=width:integer;720

a=height:integer;480i

A=fmtp:96packetization-mode=1;profile-level-id=64001e;sprop-parameter-sets=z2qahqxwwlq9pqagicba,ko4cpla=

It can be clearly seen that the RTSP protocol consists of two parts, the first part is the RTSP transport protocol, and the other part is the SDP protocol. SDP is not exactly the transport layer protocol, but planning to the Session Layer Protocol section. The SDP (sessiondescription Protocol) is a file that is generated on the server side that describes the encoding information of the media file and the link to the server in which the client sets the parameters of the playback software.

(4) RTSP

With the above analysis, we have a basic understanding of the entire structure, then there is the last part of the RTSP protocol. Real-time Streaming protocol is an application layer protocol in TCP/IP protocol system. Similar to the HTTP protocol. As for the specific structure of the agreement we put it in the back together with the code.

Here's a simple process for RTSP streaming: This is a typical request process:

1. Client->server:option request//Ask S what methods are available

1. Server->client:option Response//s Response information includes all available methods provided

2. Client->server:describe request//Requires media initialization description information provided by S

2. Server->client:describe response//s responds to media initialization description information, primarily SDP

3. Client->server:setup request//Set session properties, and transfer mode to remind S to establish session

3. Server->client:setup response//s to establish a session, return session identifiers, and session-related information

4. Client->server:play Request//C Requests Playback

4. Server->client:play response//s Back the information that should be requested

Server->client: Sending streaming media data

5. Client->server:teardown request//C to close the session

5. Server->client:teardown response//s back should request

IOS Video Live/Smart Home (line of code, zero-based) lesson:1 overall architecture