IOS live video/Smart Home (one line of code, starting from scratch) lesson: 1 overall architecture, ios Smart Home

IOS live video/Smart Home (one line of code, starting from scratch) lesson: 1 overall architecture, ios Smart Home

Some time ago, a live video/Smart Home application was required for work. This is a summary of streaming media processing in the IOS audio and video column. Here, we want to record the entire development process. On the one hand, we will discuss and learn with you, and on the other hand, we can easily check for missing information in the future.

The whole development did not use any third-party framework, and all the streaming media protocols were typed in one row. Why? It's better to teach fish to fish!

The following describes the architecture of the entire software.

We have introduced the front-end in a very detailed section in the IOS audio & video column, including detailed analysis of various processing methods including cameras, audio and video. At the backend, we also gave a detailed introduction to FFmpeg in IOS audio & video and advanced usage. We will not discuss it here. The following is a detailed analysis of the network.

(1) RTCP

RTCP: RTP Control Protocol, real-time transmission Control Protocol, which is generally used in combination with RTP. It is mainly used for data transmission monitoring and Control functions. At the same time, RTCP is based on UDP transmission. The information transmitted by RTCP is very important, including timestamp (for synchronization), serial number (for packet loss and re-sorting Detection), and load format (for data encoding format ). To put it simply, it is used for QoS feedback and synchronization of media streams. Compared with RTP, half of the bandwidth is only about 5% of RTP. Very small.

Based on the Usage Status, RTCP is divided into the following situations:

For live broadcast and smart home projects, as long as the sender report is used, we will analyze it in detail in the code below.

(2) RTP

Real-time Transport Protocol, a Real-time transmission Protocol, is generally used for multimedia data transmission. Audio and video are transmitted mainly through this Protocol. It is based on UDP protocol, which is more efficient but allows packet loss. Therefore, we need to do a lot of work during media restructuring. The RTP protocol package format is as follows:

(3) SDP

Let's take a look at the packet capture network request process:

Server Response Data. What we care about: RTSP

Response = RTSP/1.0 200 OK

CSeq: 1

Content-base: rtsp: // 192.168.36.168/

Date: July 29, 2015 GMT + 8 2:52:00

Content-Type: application/sdp

Content-Length: 424

SDP:

V = 0

O =-1804289383 1804289383 IN IP4 192.168.36.168

S = Livestream from iOS

C = INIP4 0.0.0.0

T = 00

A = control :*

M = video0 RTP/AVP 96

B = TIAS: 85528

A = maxprate: 9.0000

A = control: streamid = 1

A = rtpmap: 96H264/90000

A = mimetype: string; "video/H264"

A = framesize: 96720-480

A = Width: integer; 720

A = Height: integer; 480i

A = fmtp: 96packetization-mode = 1; profile-level-id = 64001e; sprop-parameter-sets = Z2QAHqxWwLQ9pqAgICBA, KO4CPLA =

We can clearly see that the RTSP protocol consists of two parts: the first part is the rtsp transmission protocol, and the other part is the SDP protocol. To be accurate, SDP cannot be regarded as a transport layer protocol, but it must be planned to be included in the Session Layer Protocol. SessionDescription Protocol (SDP) is a file generated by the server to describe the encoding information of the media file and the link of the server. The client uses it to set parameters of the playback software.

(4) RTSP

With the above analysis, we have a basic understanding of the entire structure, and the last part is the RTSP protocol. The real-time stream transmission protocol is an application layer protocol in the TCP/IP protocol system. Similar to Http. As for the specific structure of the protocol, we will talk about it later with the code.

Let's take a look at a simple process of RTSP Streaming Media interaction: this is a typical request process:

1. Client-> Server: OPTION request // ask S which methods are available

1. Server-> Client: OPTION response // S the response information includes all available methods provided

2. Client-> Server: DESCRIBE request // obtain the media initialization description provided by S.

2. Server-> Client: DESCRIBE response // S response media initialization description, mainly sdp

3. Client-> Server: SETUP request // sets session attributes and transmission mode to remind S to establish a session

3. Server-> Client: SETUP response // S: Creates a session and returns the session identifier and session-related information.

4. Client-> Server: PLAY request // C

4. Server-> Client: PLAY response // S. The requested information is returned.

Server-> Client: Send streaming media data

5. Client-> Server: TEARDOWN request // C request to close the session

5. Server-> Client: TEARDOWN response // S the response Request

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.