A-Gump's chocolate box: parsing the format of mobile video files 3GP/MP4

Author: King of songs

 

After completing the mobile phone player project, I have to say goodbye to the Battle of manager Zhu, which will be developed in Beijing for a few months. Prepare

After finishing the 3GP, FLV, and AVI formats, We can summarize the hard work of several months and provide some reference for later users.

 

1. Overview

All popular file formats are supported by large companies. FLV benefits from the online video sharing trend promoted by Adobe, while Avi is Microsoft's first riff, that is, video and audio are intertwined for simultaneous playback. 3GP/MP4 is the default video format proposed by Apple and supported by the ISO standard as NOKIA and other mobile phones. 3GP is a simplified version of MP4 format on mobile phones. MP4 codec combination is generally MPEG4 + AAC, 3GP is divided into 3GPP R5 (H.263/MPEG4 + AMR-NB/amr wb) by version evolution, 3GPP R6 (increased H. 264 video and aacplus audio support ).

 

Some people will mix MP4 and MPEG4. The former is the file container, and the latter is the video encoding format. The container is used to compress the encoding.

After the video and audio data are arranged as compact as possible, it is like the chocolate box of A. You don't know what is in the box, but you can follow

The established clues are used to unlock the file and retrieve the data you need.

 

File formats generally include the following three elements:

Header: Mark the file type and basic attribute information of the audio/video stream.
Index: index table. Each frame has the corresponding offset, size, and timestamp.
Stream: Real Audio and Video Stream Data.
Any file format should have the above three elements. Of course, Avi videos can be played without indexes, but they cannot be dragged and dropped. You need to re-create the indexes on your own. Solution

Demuxer finds its offset and size in the file based on frame_id, then reads it, decodes it, and plays it.

 

2. File Format Analysis

 

Next we will analyze the 3GP/MP4 file format. Apple's format has two features: 1. compact layout with almost no redundant data (AVI has many junk

Data). 2. The audio/video stream data can be stored at will without the need to be arranged in chronological order.

 

The 3GP file is composed of a series of box (atom) files. The structure of each box is 4-byte size, 4-byte type, and some data. Use

Mp4info: view the data layout of the 3GP file, for example:

 

 

For example, ftyp indicates the file version information, and mdat stores text, audio and video data. You may ask how to find the audio and video data.

What? The sub-box Trak in moov box stores the attribute descriptions of audio and video and the indexes of each sample.

 

3. About Sample Atoms

 

The bitrate attributes of video and audio (for example, video width/height, codec ID, audio Sampling Rate Audio Channels, etc.) are stored in the STSD box.

This article focuses on the five boxes of MP4 efficient compression: stts, STSS, STSC, stsz, and stco. Compared to the AVI index table, each sample has a corresponding

The efficient indexing methods of ID, flag, offset, size, and 3GP can convert Avi into MP4 at the same bit rate and reduce the file size to 20-30%!

1. stts atom (time to Sample Atoms, see the quick time format document Figure 2-28Standard document click to download): The sample is stored.

. Stts allows you to conveniently locate the corresponding sample based on Timestamp, or obtain the timestamp. stts corresponding to a sample.

Table records the number of samples with the same duration, count, and dutation.

2. STSS atom (Sync sample atom, see Figure 2-31): stores the sample ID of each key frame. STSS makes it easy to find

The key frame closest to the current frame.

3. STSC atom (sample to chunk atom): The sample is stored in the chunk to allow Optimized Data Reading. For example, the audio sample size

All are very small (AMR-Nb sample size is 32 bytes). The overhead of reading a sample each time is too large, so you can read a bunch of chunks

Sample.

4. stsz atom (sample size atom): stsz can describe the size of each sample.

5. stco atom (chunk offset atoms): stco describes the absolute offset position of each chunk in the file. The offset can be 32-bit.

It can also be 64-bit. The latter is used to support processing large files.

 

4. Use Sample Atoms to process the playback process

• Search for sample

1. determine the time, relative to the media Time Coordinate System

2. Check time-to-sample atom to determine the serial number of the sample at the given time.

3. Check sample-to-chunk atom to find the chunk of the sample.

4. Extract the trunk offset from the chunk offset atom.

5. Use sample size atom to find the offset of the sample in the trunk and the size of the sample.

 

For example, if you want to find 1st seconds of video data, the process is as follows:

1. The video data of 1st seconds is 600 relative to the time of the movie.

2. Check the time-to-sample atom, and obtain that the duration of each sample is 40, so that the 600/40 = 15 + 1 = 16 samples need to be searched.

3. Check sample-to-chunk atom to obtain the first sample of the 5th chunks. The chunk has 4 samples.

4. Check that the chunk offset atom finds that the offset of the 5th trunk is 20472.

5. Because 16th samples are the first sample of 5th trunks, you do not need to check the sample size atom. The trunk offset is the offset of the sample 20472. If it is the second sample of the trunk, find the size of the previous sample of the trunk from the sample size atom, and add the offset to get the actual location.

6. After obtaining the location, you can extract the corresponding data for decoding and playback.

• Search for key frames

The search process is similar to the sample search process. You only need to use sync sample atom to determine the sample sequence number of the key frame.

Determine the serial number of the sample at a given time
Check the sync sample atom to find the key frame after the sample sequence number.
Check sample-to-chunk atom to find the chunk corresponding to the sample.
Extract the trunk offset from Chunk offset Atom
Use sample size atom to find the offset of the sample in the trunk and the size of the sample.

5. 3GP/MP4 related resources

 

QuickTime file format specification: Click to download the most authoritative format document
Open-source 3GP/MP4 Parser: FFMPEG, gpac, helix, and Google opencore