Mobile streaming media technology and its application development direction

Address: http://www.cnblogs.com/frankboy/archive/2005/12/13/296429.html

I. Status Quo Analysis

In the value-added mobile phone business market, SMS, MMS, and color e are different from traditional media in terms of interaction and uninterrupted 24 hours, however, the transmission of static images and text content affects the full play of its media role. As the demand of end users increases, how to better integrate voice, text, and images and support multimedia functions not only gives full play to the advantages of convenient and fast text messages, but also makes up for monotonous text messages, it truly makes mobile users "deaf and deaf" and enters a beautiful world with vivid and vivid images, which has become a topic of widespread concern for mobile operators.

The emergence of streaming media has changed this situation. It does not need to download the entire file and can play while downloading the entire file to the player, realizing the dream of on-demand video on the Internet or watching movies and TVs. Now, digital media is transmitted in the form of "streams", so that people can enjoy continuous high-quality audio and video programs online in a certain bandwidth environment. In the era of Internet development, the emergence and development of streaming media technology will inevitably have a profound impact on our daily lives and work. Experts predict that streaming media will become the mainstream application on the Internet in the future. The multi-direction of communication and communication will make the communication no longer limited by time and space.
Streaming media is a way for users to download and play multimedia data over the network or a specific digital channel. One of the biggest advantages of streaming media applications is that users do not need to spend a long time downloading all the multimedia data locally before playing the video, however, you only need to download the data from the first few seconds to the local buffer to start playing the video. The data received later will be continuously input to the buffer to maintain the playing continuity, therefore, streaming media players generally only have some latency at the beginning. Streaming media systems are much more complex than download and playback systems. Therefore, we need to combine the multimedia coding/decoding and transmission technologies, this ensures stable playback quality in complex network environments.
Before transmission, multimedia data must be effectively compressed into a code stream through the encoder to reduce the usage of network resources. At present, the commonly used video encoder has MPEG-2, MPEG-4, H.261, H.263, H. 264. Window Media Video Encoder and real System Video Encoder. The encoder includes MP3, mpeg aac, Window Media Audio Encoder and Amr, and the image encoder includes JPEG and MPEG-4. The bitstream generated by the Multimedia Encoder only contains the information required to decode the bitstream. It does not contain system information such as inter-media synchronization and random access, therefore, the encoded multimedia data is also organized into a multimedia file with a specific system format for streaming media transmission or storage in the disk, currently commonly used file formats include MPEG-2 system, MP4, microsoft's ASF, real file format, QuickTime file format, and 3GPP and PP2 for 3G wireless services.
When streaming media is in real-time applications (such as live streaming media broadcast), according to the current network conditions and user terminal parameters, multimedia data is transmitted to the user by the Streaming Media Server while being encoded. In other non-real-time applications, multimedia data can be encoded in advance to generate multimedia files and stored in the disk array. When multimedia services are provided, the Streaming Media Server directly reads these files and transmits them to users. In this way, the service method has low requirements on devices. Currently, many streaming media services belong to the latter method, which requires the Streaming Media Server to have a certain mechanism to adapt to network conditions and user devices.
Currently, the adaptive bitstream module mainly adopts the following methods: converting the video bitstream in multimedia files into a specific bitrate and image size bitrate; you can also encode the content of the same video to generate multiple bitstreams with different bit rates and image sizes. Then, you can select the most suitable bitstream for transmission. The generated code streams need to be further packaged into packets of specific network transmission protocols for network transmission. Because many networks do not guarantee that the transmitted data can be received in a timely and completely correct manner, the transmitted data packets may need to be protected by forward correction encoding (FEC). After these data packets are processed, multimedia data can be transmitted to users over the network, currently, common transmission protocols include RTP/RTCP, HTTP, and MMS.
After a user receives the transmitted data, if there is a packet loss or a bit error, the error recovery process will recover the transmission error based on the additional error correction data. If the transmission error cannot be restored, the client can send a retransmission request to the server to re-transmit the lost packet before decoding. After the restored multimedia data is decoded by the decoder, the reconstructed multimedia data cannot be restored due to fault tolerance protection and data retransmission, the error concealment module can use the relevance of reconstructed multimedia data to conceal these errors, and finally play the data to the user.
Generally, server and user in the streaming media system do not communicate in one way, as mentioned above. In fact, the user end will pass a lot of feedback information to the server. For example, the capability and network connection speed of the terminal device will be transmitted to the server's code stream adaptive module to adjust the code stream, in real-time applications, this information may also be transmitted to the encoder. The packet loss rate on the user end, the time information received by the data packet, and the user buffer status will also be transmitted to the server to estimate the current network condition, to control the adaptive bitstream and data transmission policies. From the above description, the streaming media system is actually a very complex system in multimedia information processing. Currently, the main products on the market include Windows Media of Microsoft, real's real system and Apple's QuickTime, among which Windows Media has the largest market share.

II. Key Technologies of streaming media

the key technology for implementing streaming media is stream transmission. Streaming Transmission is widely defined. It refers to the technology that transmits media (such as video and audio) over the network.
stream transmission is divided into Sequential stream transmission and real-time stream transmission:
ordered stream transmission adopts the sequential download mode. When downloading files, you can watch online programs at a given time, users can only view the downloaded part, but cannot jump to the part that has not yet been downloaded. This method is not like Real-Time Streaming Transmission, and can be adjusted according to the user connection speed during transmission. Ordered stream transmission is not suitable for long clips and video programs with random access requirements, such as lectures, speeches, and Demos. It also does not support on-site broadcast. Strictly speaking, it is a VOD technology.
real-time stream transmission ensures that the media signal bandwidth matches the network connection and allows you to watch programs in real time. Unlike HTTP streaming, Real-Time Streaming requires a dedicated Streaming Media Server and transmission protocol. Real-time stream transmission is always transmitted in real time, which is especially suitable for on-site events and supports random access. Users can fast forward or backward view content. Theoretically, real-time streams cannot be stopped once played, but can be paused periodically.
the stream transmission mode generally uses the RTP/udp and RTSP/tcp communication protocols to establish a connection with the/V (audio/video) server, redirects the server output to the destination address of the client that runs the/V player Program . As shown in figure 1, a set of dedicated servers and players are usually configured for the stream transmission system.

1. Real-time transmission protocol RTP and RTCP
RTP (Real-Time Transport Protocol) is a transmission protocol for multimedia data streams over the Internet. It works for one-to-one or one-to-many transmission, provides time information and enables stream synchronization. RTP generally uses UDP to transmit data, and can also work over TCP or ATM. When an application starts an RTP session, it uses two ports, one for RTP and the other for RTCP. RTP itself does not provide a reliable transmission mechanism for transmitting data packets in order, nor provides traffic control or congestion control, but relies on RTCP to provide these services. Usually RTPAlgorithmIt is not implemented as an independent network layer, but as an applicationCode.
RTCP (Real-Time Transport Control Protocol) and RTP provide traffic control and congestion control services. During the RTP session, participants periodically transmit RTCP packets, which contain statistics such as the number of sent packets and the number of lost packets. The server can dynamically change the transmission rate based on these statistics, even change the payload type. The combination of RTP and RTCP can effectively provide feedback to reduce overhead and improve transmission efficiency. It is very suitable for transmitting real-time online data.

2. Real-time stream protocol RTSP
Real-Time Streaming Protocol RTSP (Real-Time Streaming Protocol) is a protocol jointly proposed by RealNetworks and Netscape. It defines how to enable one-to-multiple applications to effectively transmit multimedia data over an IP network. RTSP is based on RTP and RTCP in the architecture. It uses TCP or RTP for data transmission. Compared with HTTP, RTP transmits multimedia data while HTTP transmits HTML data. When RTSP is used, both the client and server can send requests, that is, RTSP can provide two-way services, while HTTP requests are sent by the client and the server responds.

3. Resource Reservation Protocol (RSVP)
Audio and video data streams are more sensitive to network latency than data services. In addition to bandwidth requirements, other conditions are required to transmit high-quality audio and video information over the network. Resource Reservation Protocol (RSVP) is an Internet Resource Reservation Protocol under development. It provides QoS for streaming media transmission by reserving some network resources (bandwidth. Some experimental systems, such as the network video conferencing tool Vic, have integrated RSVP.
3GPP UMTS video media codec technical specification is ITU-T H.263 profile 0 level 10, is also a must use the video decoder. In addition, the PSS should support H.263 profile 3 level 10 decoder and MPEG-4 Visual Simple Profile level 0 decoder, two video decoders are optional in practical use. The recently proposed H.264 standard has also aroused widespread interest in the industry. 3GPP PSS R6 is also actively considering incorporating it into specifications.

Mobile streaming media player:
Mobile streaming media terminals all adopt h264 Video Compression Algorithm for optimized acceleration, so that low-bandwidth encoding (15-25 Kpbs) suitable for wireless transmission can transmit better quality images, or use less bandwidth to transmit videos of the same quality. Major manufacturers with these technologies, such as Microsoft and real, are only developing applications on special video mobile phones. Therefore, for application developers, the player needs to be transplanted on various mobile terminals. Main platforms include:
Pocket PC Platform
Dopod 686/696
Lenovo et180/560
Daxian cu928/eTEN p300b
Symbian Platform
Nokia 7650/6600/3650/7610/nokia6260/6630/7610/6620/3620/3660/3600/3650/n-gage/
Soo P802/p908
Simens SX1/SENDO x
Panasonic x700/
Samsung SGH-D710
Linux platform
Motorola a760
Smartphone platform
Dopod515/535
Moto8380/8390

Iii. main applications of mobile streaming media
(1) Information Service
Including financial information, news, instant sports broadcast, weather information and other services. Users only need to access the portal to obtain a large amount of information, or use the Information Push Service through subscription. Information content can be provided as streaming media.
(2) Entertainment Services
This includes downloading, playing, and online playing of cartoons, audios, videos, and highlights of TV programs. It also provides services such as mobile games and TV watching on mobile phones.
(3) Communication Service
Including multimedia messages and video calls/Conferences containing streaming media content, which makes communication more convenient and colorful.
(4) Monitoring Service
It mainly includes traffic monitoring and Home monitoring. Traffic Monitoring allows the transportation department to view the traffic conditions of the expressway and main roads in real time, view the traffic conditions of the specified road section, and check the traffic conditions of each road section through the location service on the way. Home monitoring can monitor home and office information in real time. You only need to install a Web-based digital video camera and connect to the Internet to monitor your home or office on a mobile terminal or PC.
(5) Positioning Service
Provides map and Wizard services. You can preview scenic spots, book hotels, and movie tickets. In the next few years, the mobile streaming media business will be greatly developed and will be gradually implemented with the continuous development of networks and terminals.

Iv. Development and restrictions of mobile streaming media

The development of mobile streaming media businesses has brought new hopes for mobile value-added services. The development of 2.5g, 3G, and over 3G wireless networks also enables streaming media technology to be used on wireless terminal devices, currently, China Unicom provides CDMA 1X with a network bandwidth of up to 100 kbit/s, which is sufficient to provide qcif streaming media services. With the application of 3G wireless networks, the user's network bandwidth can be 384 kbit/s. On the other hand, mobile devices have higher computing power and larger storage space. Needless to say, high-end mobile phones such as smart phone and Pocket PC are normal mid-range mobile phones, such as Nokia 6610, which can also achieve basic H. 264 of software decoding.
Streaming media applications for wireless networks pose greater challenges to current coding and transmission technologies. First, compared with wired networks, wireless networks are not stable, apart from the transmission rate fluctuation caused by network traffic, the Mobile speed and location of handheld devices will seriously affect the transmission rate. Therefore, efficient adaptive coding technology is crucial. Secondly, the wireless channel environment is much worse than the wired channel, and the data bit rate is much higher. The highly compressed code stream is very sensitive to transmission errors, it will also cause the spread of incorrect and backward images. Therefore, the wireless streaming media requires a good Error Tolerance Technology in source and channel coding. During the development of mobile streaming media businesses, the following problems exist:
(1) narrow wireless network bandwidth and severe interference
As the mainstream 2G and 5G wireless network technologies of China Unicom and China Mobile, respectively, CDMA 1X and GPRS have greatly improved network transmission bandwidth, but they are still very limited. In the case of theoretical peaks, the download transmission rate of CDMA 1X reaches 144 kbps, but in actual conditions, the stable transmission rate is usually around 70 kbps. GRPS can theoretically reach 115 kbps, but in practice, the stable transmission rate is usually around 20 kbps. With the increase of users, the network performance will be further reduced. In addition, due to serious interference from wireless networks, the possibility of network transmission error codes is greatly increased.
(2) Low processing capability and small memory capacity of Mobile Terminals
Although there are more and more high-end mobile phones in the Chinese market based on arm9-or chips with the same capabilities, the huge market opportunities brought by the large base of low-end users in mobile phones, this makes various terminal manufacturers pay special attention to low-end users. Therefore, the most popular mobile phones that currently occupy the market share are still using the ARM7 chip, with dozens of MIPS processing capabilities.
At present, the memory capacity of mobile terminals is usually relatively limited. In the market, the mainstream brew mobile phones usually reserve about KB of dynamic memory for applications, while the dynamic memory reserved for applications based on j2-based mobile phones is usually 64 kB or KB; the dynamic allocation reserved for applications based on high-end mobile phones such as Symbian, Linux, and Windows Mobile is about 1-4 MB.
(3) Diversified terminal system platforms and LCD
Compared with PC platforms, mobile terminal system platforms are more diversified. Common system platforms include Symbian, Linux, Windows Mobile, Palm OS, and some private platforms. The diversity of mobile terminal systems will continue to exist for a long time. In order to provide a unified environment for running mobile apps, the emergence of j2s and brew came into being. However, different manufacturers usually have different support for j2s and brew. The diversity of platforms, coupled with different LCD sizes, makes it very difficult to adapt to a variety of mobile terminal applications.
(4) limited battery energy for mobile terminals
Despite the increasing computing power of mobile phone devices, because it is powered by batteries, codec processing cannot be too complicated, in addition, it is best to adjust the receiving and processing of streaming media based on the battery of user equipment. Energy Management Technology is also a hot research topic of mobile streaming media.