Android uses mediarecorder to monitor MPEG4 videos

It took a whole week to finally break through the technical difficulties. It seems that there is less discussion on the Internet.

The main function that needs to be implemented is to collect videos in real time on Android phones and display them in real time remotely, such as on a PC. The Android mobile phone is used as the monitoring camera.

At first, I found an open-source project of smartcam. After reading the source code, I found that the implementation principle is to use the android camera image pre-collection and implement the onpreviewcallback callback function onpreviewframe, after obtaining the original image data collected by camera, it is compressed into JPEG format and uploaded to the PC end. The PC end is connected to the received JPEG Image Sequence for real-time decompression and display, achieving the expected results.

Although this method is a little clumsy, it is acceptable. However, it is unacceptable that JPEG is only compressed within the frame. The image sequence of 320x280 is about 10 FPS, and the network traffic reaches kb/s. This is almost impossible for practical application.

Therefore, the video stream must be directly transmitted in MPEG4 or H.264 format. It seems that my development machine (HTC G8) only supports MPEG4, so MPEG4 is selected. But how to collect video streams in real time is a big problem. After all, in terms of video, Android does not provide a callback function similar to onpreviewframe.

If you want to use opencore or a newer stagefright, you may be discouraged immediately after looking at the SDK framework. This is too huge. In addition to the local file path, you can also bind a socket port to the connector (actually called file descriptor. That is to say, you can obtain the video stream data of mediarecorder in real time through a socket port.

(In fact, the content of the above blog can be found in the videocamera file of the open-source project sipdroid, but I am very grateful to the blog host zhangzhenj for answering questions from netizens .)

The video stream received through the socket is a little different from the video stream data directly written in the local file. Because the video is transmitted through the socket, the video file cannot be written back, mediarecorder usually writes back the video file when it ends the video, so that the video can be played by the player. Therefore, the data received through the socket cannot be played after it is saved. After checking the output file in hexadecimal mode, it is found that the first 32 bytes are 00, followed by mdat. The problem occurs here: a ftyp box description (28 bytes) and a mdat length description (4 bytes) are missing ). someone has successfully solved this problem on the Internet. Find the starting position of moov in the data and find that there is a description of ftyp in front of it. The length is just 28 bytes. You can copy this 28bytes to the 28byte at the beginning of the file. The ftyp description is from the starting position of moov.
The first 32byte of the parameter starts until the first 4 byte (the last 4 byte is the moov length description ). Then the mdat length is the starting position of moov minus 0x20. Then write the value to the first 4 bytes of mdat. It just fills up 32 bytes, and then the video can be played smoothly.

After the saved file can be played, how can I display the video stream in real time? Check the file format of MPEG4 and you will soon find the answer. The answer is in mdat. After mdat, the video media data is followed. Each frame starts with 00 00 01 B6. It seems that there is no ending mark. It is estimated to be used for frame splitting. The two bits after the start sign are the I, P, and B frames. The corresponding values are respectively and 10. Remember that the two bits are not two bytes.

Okay, you can use FFMPEG to decode one frame of mdat and then display it. This method is feasible, but the details are still a little troublesome, the key is that FFMPEG must specify the width and height when decoding MPEG4; otherwise, the decoding fails.

The idea is like this, and the complete code is not available yet.

Tip: http://www.linuxidc.com/Linux/2011-04/34467.htm