How to extract and generate mp4 files in Android

How to extract and generate mp4 files in Android
With the gradual popularization of Android 4.4 and later versions, the MediaExtractor class introduced by Android 4.1 and the MediaMuxer class introduced by Android 4.3 can finally begin to officially "glow and Heat. MediaMuxer is mainly used to generate multimedia files (such as mp4 files) by mixing audio and video data, while MediaExtractor is the opposite. It is mainly used to separate audio and video data of multimedia files. This article describes how to use the MediaExtractor and MediaMuxer classes provided by the Android SDK to extract and generate mp4 files, point out the pitfalls encountered during development, and provide simple Demo code. Demo target: extract the video data in the input.mp4 file and generate the pure video output.mp4 file behind the audio and video data. The Android SDK has few introductions about these two classes. Therefore, before giving a demo, let's take a look at these two classes. 1. mediaExtractor is mainly used for separation of audio and video hybrid data. The interface is relatively simple. First, you must use the setDataSource (String path) function to set the data source. The data source can be a local file address, you can also use the network bitstream address of the HTTP protocol. Then, you can use the following code block to obtain detailed information about the code stream, such as MimeType, resolution, encoding format, bit rate, and frame rate.

Int videoTrackIndex =-1; int audioTrackIndex =-1; for (int I = 0; I <mMediaExtractor. getTrackCount (); I ++) {// obtain the detailed format of the bitstream/configuration information MediaFormat format = mMediaExtractor. getTrackFormat (I); String mime = format. getString (MediaFormat. KEY_MIME); if (mime. startsWith ("video/") {videoTrackIndex = I;} else if (mime. startsWith ("audio/") {audioTrackIndex = I ;}....}

 

After obtaining the details of the media file, you can select the specified channel and separate and read the data:

MMediaExtractor. selectTrack (videoTrackIndex); // select to read video data while (true) {int sampleSize = mMediaExtractor. readSampleData (buffer, 0); // read a frame of data if (sampleSize <0) {break;} mMediaExtractor. advance (); // move to the next frame} mMediaExtractor. release (); // remember to release resources after reading

 

2. The MediaMuxer class is mainly used to generate multimedia files by mixing audio and video files. To create such objects, You need to input the output file location and format. The constructor is as follows:

public MediaMuxer(String path, int format);

 

After an object is created, an important operation is addTrack () to add a data channel. This function needs to input a MediaFormat object. MediaFormat is the media format class used to describe the format parameters of the media, such as video frame rate and audio sampling rate. In this example, you can directly use MediaExtractor. getTrackFormat () to parse the MediaFormat object. If you want to create this MediaFormat object yourself, you can use the following static method to create this class:

MediaFormat format = MediaFormat.createVideoFormat("video/avc",320,240);

 

Note that there is a big pitfall here, that is, if you manually create a MediaFormat object, remember to set the "csd-0" and "csd-1" parameters:

byte[] csd0 = {x,x,x,x,x,x,x...}byte[] csd1 = {x,x,x,x,x,x,x...}format.setByteBuffer("csd-0",ByteBuffer.wrap(csd0));format.setByteBuffer("csd-1",ByteBuffer.wrap(csd1));

 

What is "csd-0" and "csd-1", for H264 video, it corresponds to sps and pps, for AAC audio, it corresponds to ADTS, people who develop audio and video should know that it is generally stored in IDR frames generated by encoder. After adding a data channel through addTrack (), record the trackIndex returned by the function, and then you can call MediaMuxer. writeSampleData () to write data to the mp4 File happily. The second pitfall is generated here, that is, the last parameter of the writeSampleData function is a BufferInfo object. You must enter the correct value carefully:

BufferInfo info = new BufferInfo();info.offset = 0;info.size = sampleSize;info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;info.presentationTimeUs = timestamp;

 

Where, info. size must be filled in the data size info. flags needs to indicate whether it is a synchronous frame/Key Frame info. presentationTimeUs must provide the correct timestamp. Note that the unit is us. For example, for videos with a frame rate of x f/s, the timestamp interval is 1000/x MS skipping these pits, You can smoothly complete the mp4 file write, also, after completion remember to close and release resources:

mMediaMuxer.stop();mMediaMuxer.release();

 

3. With the simple introduction and foreshadowing above, it is not difficult to understand the demo code. Note for running the demo code: (1) Android 4.3 and above (2) copy the input.mp4 file to the core part of the sdcard Code as follows:

  protected boolean process() throws IOException {      mMediaExtractor = new MediaExtractor();                mMediaExtractor.setDataSource(SDCARD_PATH+"/input.mp4");                                     int mVideoTrackIndex = -1;      int framerate = 0;      for(int i = 0; i < mMediaExtractor.getTrackCount(); i++) {          MediaFormat format = mMediaExtractor.getTrackFormat(i);          String mime = format.getString(MediaFormat.KEY_MIME);          if(!mime.startsWith("video/")) {                              continue;          }          framerate = format.getInteger(MediaFormat.KEY_FRAME_RATE);                      mMediaExtractor.selectTrack(i);          mMediaMuxer = new MediaMuxer(SDCARD_PATH+"/ouput.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4);          mVideoTrackIndex = mMediaMuxer.addTrack(format);            mMediaMuxer.start();      }             if(mMediaMuxer == null) {          return false;      }             BufferInfo info = new BufferInfo();      info.presentationTimeUs = 0;      ByteBuffer buffer = ByteBuffer.allocate(500*1024);              while(true) {          int sampleSize = mMediaExtractor.readSampleData(buffer, 0);          if(sampleSize < 0) {              break;          }          mMediaExtractor.advance();          info.offset = 0;          info.size = sampleSize;          info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;                  info.presentationTimeUs += 1000*1000/framerate;          mMediaMuxer.writeSampleData(mVideoTrackIndex,buffer,info);      }      mMediaExtractor.release();             mMediaMuxer.stop();      mMediaMuxer.release();             return true;  }

 

4. this is a summary of how to extract and generate mp4 files in Android. If you have any questions or suggestions, please leave a message or send a letter to [email protected, or follow my Sina Weibo @ Lu _ Jun to get the latest articles and information.