Using MP4V2 to synthesize h264+aac mp4 files

using MP4V2 to synthesize h264+aac mp4 files

Recording program to add new features: Record CMMB TV shows, our cards are sent out of the RTP stream (H264 video and AAC audio), the recording program to do the work is:

(1) Receiving and parsing RTP packets, separating out H264 and AAC data streams;

(2) H264 video and AAC audio are encapsulated in some format and finally saved as a file for viewing by the user.

The first step is already part of the code to refer to, so it's done quickly.

In the second step, we decided to encapsulate the MP4, and after finding some information, we decided to use the Open Source Library mp4v2 to synthesize the MP4 file.

The technical route has been determined and the work is started.

First, basic knowledge of the MP4 format.

About MP4 format, the online introduction of many, there are the following content for reference:

(1) Two ISO standards:

[ISO/IEC 14496-12]:iso base media file format--"is a general format forming the basis for a number of and more specific File formats. This format contains the timing, structure, and media information for timed sequences of media data, such as audio-visual Presentations "

[ISO/IEC 14496-14]:mp4 file Format--"This specification defines MP4 as an instance of the ISO Media file format [ISO/IEC 14496-12 and ISO/IEC
15444-12]. "

Defines the MP4 file format standard.

(2) http://wenku.baidu.com/view/673482284b73f242336c5f4c.html

Is the above two standard explanation, suggest to see this first, understand probably, concrete detail again look ISO standard document.

Second,Technical verification. 

The main thing is to write the verification code to verify the technical feasibility.

To the official website to download mp4v2 source code, compile, installation process is not mentioned. All information can be found at http://code.google.com/p/mp4v2/.

The first part of the verification code, finished quickly, but the encapsulated file has a problem, unable to play.

The composition section, the code is as follows:

1 static void * writeThread (void * arg)
 2 {
 3 rtp_s * p_rtp = (rtp_s *) arg;
 4 if (p_rtp == NULL)
 5 {
 6 printf ("ERROR! \ N");
 7 return;
 8     }
 9 
10 MP4FileHandle file = MP4CreateEx ("test.mp4", MP4_DETAILS_ALL, 0, 1, 1, 0, 0, 0, 0); // Create mp4 file
11 if (file == MP4_INVALID_FILE_HANDLE)
12 {
13 printf ("open file fialed. \ N");
14 return;
15}
16
17 MP4SetTimeScale (file, 90000);
18
19 // Add h264 track
20 MP4TrackId video = MP4AddH264VideoTrack (file, 90000, 90000/25, 320, 240,
21 0x64, // sps [1] AVCProfileIndication
22 0x00, // sps [2] profile_compat
23 0x1f, // sps [3] AVCLevelIndication
24 3); // 4 bytes length before each NAL unit
25 if (video == MP4_INVALID_TRACK_ID)
26 {
27 printf ("add video track failed. \ N");
28 return;
29}
30 MP4SetVideoProfileLevel (file, 0x7F);
31
32 // Add aac audio
33 MP4TrackId audio = MP4AddAudioTrack (file, 48000, 1024, MP4_MPEG4_AUDIO_TYPE);
34 if (video == MP4_INVALID_TRACK_ID)
35 {
36 printf ("add audio track failed. \ N");
37 return;
38}
39 MP4SetAudioProfileLevel (file, 0x2);
40
41
42 int ncount = 0;
43 while (1)
44 {
45 frame_t * pf = NULL; // frame
46 pthread_mutex_lock (& p_rtp-> mutex);
47 pf = p_rtp-> p_frame_header;
48 if (pf! = NULL)
49 {
50 if (pf-> i_type == 1) // video
51 {
52 MP4WriteSample (file, video, pf-> p_frame, pf-> i_frame_size, MP4_INVALID_DURATION, 0, 1);
53}
54 else if (pf-> i_type == 2) // audio
55 {
56 MP4WriteSample (file, audio, pf-> p_frame, pf-> i_frame_size, MP4_INVALID_DURATION, 0, 1);
57}
58
59 ncount ++;
60
61 // clear frame.
62 p_rtp-> i_buf_num--;
63 p_rtp-> p_frame_header = pf-> p_next;
64 if (p_rtp-> i_buf_num <= 0)
65 {
66 p_rtp-> p_frame_buf = p_rtp-> p_frame_header;
67}
68 free_frame (& pf);
69 pf = NULL;
70
71 if (ncount> = 1000)
72 {
73 break;
74}
75}
76 else
77 {
78 // printf ("BUFF EMPTY, p_rtp-> i_buf_num:% d \ n", p_rtp-> i_buf_num);
79}
80 pthread_mutex_unlock (& p_rtp-> mutex);
81 usleep (10000);
82}
83
84 MP4Close (file);
85} 

Phenomenon: No image, no sound, no playback at all.

So, the hard work began: tracking the cause of the search.

(1) using VLC to play the synthesized MP4 file, see the detailed output:

1 VLC-VVV test.mp4
2 [0x8e9357c] mp4 stream debug:found Box:ftyp size 
3 [0x8e9357c] mp4 stream Debug:found Box:free size 136 
4 [0x8e9357c] mp4 stream debug:skip box: "Free" 
5 [0x8e9357c] mp4 stream Debug:found Box:mda T size 985725 
6 [0x8e9357c] mp4 stream debug:skip box: "Mdat" 
7 [0x8e9357c] mp4 stream Debug:found box:moov siz e 5187 
8 [0x8e9357c] mp4 stream debug:found box:mvhd Size 108 

You can see that VLC (actually called Libmp4 Library) parses the box correctly, and the size of the Mdat is correct.
But the next line:
Skip box: "Mdat"
It is more strange, clearly resolved correctly, why should mdat ignore it. You know, the real audio and video data is stored in the Mdat. If Skip is dropped, there is no data behind the decoding, of course it can't be played.

(2) If you find doubt, continue to follow.

View VLC source code, found in file modules/demux/mp4/libmp4.c: Skip information is printed by the Mp4_readboxskip () function, and the call place in libmp4.c 2641 lines: