Implementation and Research of RTSP-based on-demand mobile phone video (extended support for Android and iOS platforms)

With the arrival of 3G, the bandwidth is higher and the traffic fee is lower, and multimedia applications such as mobile phone and TV will surely develop greatly. I will sum up my previous experience, I will discuss with you how to create a VoD solution for mobile phones, and finally provide a preliminary client implementation result. Welcome to the discussion.

First, let's talk about the architecture. For ease of management and expansion, bandwidth restrictions, and multi-user concurrency, commercial solutions will adopt the Streaming Media Server + Web Server + Transfer Server + mobile client solution.

The streaming server collects video sources, compresses the encoding, and waits for RTSP connection requests from clients at any time;
Web servers facilitate the publishing and Management of video information;
The Transmission Server is optional. It is used to forward RTSP requests from the client to the server and transfer the server to the client in real time, the advantage is that more users can watch videos at the same time under the same bandwidth;
The mobile client can use a built-in player (such as the RealPlayer on Nokia) or an independent player developed by itself. The advantage of the former is to lower the threshold for users and facilitate large-scale applications; the latter is easy to expand and customize to meet more features.
Streaming Server is the core of the entire solution. Currently, the mainstream Streaming Media Server solutions are as follows:

Helix Server: with the strength of real, this is the most popular solution currently. It supports all audio and video formats with stable performance. It is the only solution that can span windows Mac, Linux, Solaris, the streaming media service platform for HP/UX users. It supports player playback on mobile phones. The free version of Helix Server only supports 1 Mbit/s of traffic, and the Enterprise Edition is very expensive. Of course, it is another thing to crack :)
Darwin server: this is an open-source Streaming Media Solution launched by Apple. The supported formats are not as many as helix. However, because they are open-source and free, they have a lot of development space for developers.
Live555 Media Server: stable performance, but supports few formats (only MP3, Amr, AAC, MPEG4 es, and other streams). It is rarely used independently and generally used as part of the system.
Windows Media Server: only available on the Microsoft platform.
The mobile phone Framework process is as follows:

 

 

Currently, there are two transmission protocols for mobile clients and servers: HTTP and RTSP. In the early days, mobile phones and TVs often use HTTP. HTTP has the advantages of no special server software or IIS, firewall Nat does not need to be considered, but HTTP does not support real-time streaming and bandwidth is wasted. RTSP is the mainstream standard for streaming media transmission. Even Microsoft has abandoned MMS and switched to support RTSP, RTSP supports pause, pause, and stop operations on the client. You do not need to consider audio/video synchronization issues (because the audio and video are read from different RTP ports for buffering ). It is worth noting that after the RTSP is successful, RTP transmission starts, which can be divided into RTP over TCP and RTP over UDP. The former ensures that each packet can be received and re-transmitted if it is not received, in addition, you do not need to consider the firewall Nat. The latter only guarantees the best transmission effort, and does not re-transmit frames, so the real-time performance is good, and the firewall Nat problem should be solved. If you want to use a mobile TV with a high frame rate, we recommend that you use UDP for transmission, because the retransmission of data with a high latency is meaningless to users and would rather discard it.

I used the powerful open-source library live555 in the network to implement the RTSP/RTP protocol. Its performance is stable and it supports transmission of most audio and video formats. (Of course, FFMPEG also implements the network transmission part, which can also be used after modification) after the live555 is cropped and transplanted to Symbian and Windows Mobile, this part of work is time-consuming in Symbian real machine debugging.

Of course, FFmpeg is used for video decoding, and MPEG4 SP/H is transplanted. the 264 decoder supports 32 K, CIF, and 5-10fps without any optimization, which is sufficient for general streaming media applications. It will also be optimized by algorithms and Assembly later. After decoding, you also need to pass through yuv2rgb and scale. Note that the decode of FFMPEG has a deactivating zone, that is, the linesize of the qcif image is 176 instead of 192, if you find that the decoded image is green, you need to convert it with img_convert () (the destination format is pix_fmt_yuv420p ). In Symbian, use DSA to directly write the screen. Windows
SDL.

Audio Decoding mainly includes AAC, amrnb, and amrwb. AAC and amrnb are audio supported by GPRS and edge bandwidth (AAC is better than amrnb), and amrwb is the 3G audio format. In FFMPEG 0.5 release, fixed point Decoding of amrnb/WB is supported, which is very powerful.

 
Both Symbian and Windows Mobile are tested. The effects of the 6122c and Windows mobile5.0 simulators are as follows:

 

 

The demo video address is rtsp: // v. starv. TV/later.3gp, the video is MPEG4 sp, and the audio is amrwb. Currently, only images can be seen, and audio is not added yet.

Note that the access point for streaming media applications is generally cmnet, and cmwap is only used to browse low-volume applications such as web pages.

The program is still further improved, interested can communicate with me, lius1984@gmail.com

 

2010.05.15: The program is still being improved. Currently, mobile browsers and players are supported on s60 3rd/5th, and h264/MPEG4, AAC/amrnb are supported.

Currently, WM, Symbian, and MTK are supported.

The latest supported Android and iOS platforms. See http://blog.csdn.net/lius1984/article/details/7669631