Mobile Android audio and video collection and live video push, achieving individual and mobile monitoring applications

It coincides with the 2014 Google I/O conference. It is not difficult to see that android has been quickly extended to all aspects of life with the advancement of Google and its openness. from Android smartphones and tablets, to the wearable Android Ware, glasses, watches, Android cars, smart homes, TVs, and even recent news, several Japanese robots are all Android systems, focus on the monitoring industry, mobile terminal devices in traditional monitoring, such as individual soldier devices, handheld devices, and on-board terminal devices, including smart devices used in home monitoring, can be replaced by the Android system, which is not only easy to develop, but also easy to expand, and more intelligent devices.

Figure-Android applications in handheld devices

Step by step, we first implement the real-time monitoring function for audio and video collection and uploading of Android phones. There is an open-source project on Google Code:Spydroid-ipcameraSpydroid can create a simple HTTP Server and RTSP Server on the Android mobile phone. The function is similar to the general IpCamera. It can access the camera and modify the monitoring configuration through the webpage, you can also obtain real-time monitoring audio and video through http or rtsp. The code structure shows that, spydroid has implemented RTSPServer, RTSPClient, RTP, RTCP, H264, AAC... and so on. In short, all the Utility we need is ready. We only need to combine these functions to meet our live broadcast needs.

Figure-IPCamera implemented by Android

In the previous blog"Distributed live streaming media server system based on Darwin", We use a Windows camera and microphone to collect audio and video data, encode it into H264 and AAC, and then pack it with RTP and push it to the Darwin Streaming Media Server for live broadcasting. Similarly, we refer to the EasyIpCamera design method. After the App is started, it connects to the central management server and receives control commands from CMS. The android camera video and mic sound are collected for H264 and AAC encoding (Here we only refer to spydroid for hard encoding, and soft encoding will be updated later, but after trying, currently, most Android audio and video collections support hard encoding). Then, RTSP and RTP are used to push real-time audio and video data to the Streaming Media Server, which forwards and distributes the data to achieve live broadcasting.

Figure-connect Android to EasyDSS

This is mainly the push process of RTSP/RTP, which is in the previous blog"DSS-based streaming media forwardingThis process is described in detail. The specific push code can also be referred to the DarwinInjector class implementation of live555, here, we can directly modify the RTSPClient in spydroid to implement the ANNOUNCE/SETUP/PLAY/RTP process. For the specific process source code and Protocol process, refer to"Live Video framework implemented with Darwin and live555.

Android acquisition side download: http://pan.baidu.com/s/1kTwrasB EasyDSS (Win) Server Download: http://pan.baidu.com/s/1ntoFSSh EasyDSS (Linux) Server Download: http://pan.baidu.com/s/1c0b6bUo

Usage and blog"Distributed live streaming media server system based on DarwinThe EasyIpCamera method in is the same. You only need to configure the CMS address and port number of the EasyDSS server to access EasyDSS. All subsequent processes are similar to EasyIpCamera, audio and video streams are pushed only when a client requests real-time videos. In other cases, they are only connected to CMS, and the streaming media is in the Idle state.

Figure-Android collection end configuration page

Real-time effect: after testing, the latency is highly correlated with the network and mobile phone. Different network conditions and mobile phone hardware may have different latency effects. Samsung Glaxy3 mobile phone is used, in the case of good network conditions, the latency of real-time audio and video is about 1.3 s (transmitted over the Internet:

Figure-continuous test 2 '52 'with a latency of 1.35 s

Figure-continuous test 9 '17' with a latency of 1.58 s

In the future, we will continue to expand and optimize Android real-time audio and video collection programs, including video recording, captured images, soft encoding, intercom, jitter control, and video upload. You are welcome to share your ideas and progress!