Android uses MEDIACODEC to implement video hard decoding

http://blog.csdn.net/halleyzhang3/article/details/11473961

Http://www.360doc.com/content/14/0119/10/8122810_346350456.shtml

Android uses MEDIACODEC to implement video hard decoding

This article tells you how to use the Android standard API (MEDIACODEC) for hardware encoding and decoding of video. The routine starts with the camera capture video, then H264 encoded, then decoded, and then displayed. I'll try to be brief and clear, not to show the irrelevant code. However, I do not recommend that you read this article, and do not recommend that you develop such applications, but should instead develop some poke fish, birds, enjoyable program. Well, the following is for those who are stubborn, after reading maybe you will agree with me: Android is just a toy, it is difficult to rely on it to do a reliable application.

1. Capturing video from the camera

Video data can be obtained through the callback of the camera preview.

First create the camera and set the parameters:

1Cam =Camera.open (); 2 Cam.setpreviewdisplay (holder); 3Camera.parameters Parameters =cam.getparameters (); 4Parameters.setflashmode ("Off");//No Flash5 parameters.setwhitebalance (Camera.Parameters.WHITE_BALANCE_AUTO); 6 Parameters.setscenemode (Camera.Parameters.SCENE_MODE_AUTO); 7 Parameters.setfocusmode (Camera.Parameters.FOCUS_MODE_AUTO); 8 Parameters.setpreviewformat (IMAGEFORMAT.YV12); 9 parameters.setpicturesize (Camwidth, camheight); Ten parameters.setpreviewsize (Camwidth, camheight);  One     //these two properties will be error if these two properties are set differently than the real phone ACam.setparameters (parameters);

           The width and height must be the size supported by the camera, or the error will be. To get all the supported dimensions, use getsupportedpreviewsizes, which is no longer described here.    It is said that all parameters must be set up, missing one can be an error, but it is said that I only set a few properties are not wrong. Then I started the preview:

1 New byte [Camwidth * camheight * 3/2];   2 Cam.addcallbackbuffer (BUF);   3 cam.setpreviewcallbackwithbuffer (this);            4 Cam.startpreview ();   

 Setpreviewcallbackwithbuffer is necessary, or it will be inefficient to reallocate buffers every time the callback system is reassigned.

The original picture can be obtained in onpreviewframe (of course, this must be implements Previewcallback). Here we pass it to the encoder:

1  Public void onpreviewframe (byte[] data, Camera camera) {  2     ifnull  ) {  3         framelistener.onframe (data, 0, data.length, 0);   4     }  5    cam.addcallbackbuffer (BUF);   6 }  

2. Coding

The first thing to initialize the encoder:

1Mediacodec = Mediacodec.createencoderbytype ("VIDEO/AVC"); 2Mediaformat Mediaformat =Mediaformat.createvideoformat (type, width, height); 3Mediaformat.setinteger (Mediaformat.key_bit_rate, 125000); 4Mediaformat.setinteger (Mediaformat.key_frame_rate, 15); 5 Mediaformat.setinteger (Mediaformat.key_color_format, MediaCodecInfo.CodecCapabilities.COLOR_  Formatyuv420planar); 6Mediaformat.setinteger (Mediaformat.key_i_frame_interval, 5); 7Mediacodec.configure (Mediaformat,NULL,NULL, Mediacodec.configure_flag_encode); 8Mediacodec.start ();

 
And then he feeds the data, and the data comes from the camera:

1  Public voidOnframe (byte[] buf,intOffsetintLengthintflag) {  2bytebuffer[] Inputbuffers =mediacodec.getinputbuffers (); 3bytebuffer[] Outputbuffers =mediacodec.getoutputbuffers (); 4    intInputbufferindex = Mediacodec.dequeueinputbuffer (-1); 5    if(Inputbufferindex >= 0)  6Bytebuffer InputBuffer =Inputbuffers[inputbufferindex]; 7 inputbuffer.clear (); 8 inputbuffer.put (buf, offset, length); 9Mediacodec.queueinputbuffer (inputbufferindex, 0, length, 0, 0); Ten    }   OneMediacodec.bufferinfo Bufferinfo =NewMediacodec.bufferinfo ();  A    intOutputbufferindex = Mediacodec.dequeueoutputbuffer (bufferinfo,0);  -     while(Outputbufferindex >= 0) {   -Bytebuffer OutputBuffer =Outputbuffers[outputbufferindex];  the        if(Framelistener! =NULL)   -Framelistener.onframe (OutputBuffer, 0, length, flag);  -Mediacodec.releaseoutputbuffer (Outputbufferindex,false);  -Outputbufferindex = Mediacodec.dequeueoutputbuffer (bufferinfo, 0);  +}

First, the data from the camera is fed to it, and then the compressed data from it is fed to the decoder.

3. Decoding and display

First, the initial resolution code device:

1 mediacodec = Mediacodec.createdecoderbytype ("VIDEO/AVC");   2 Mediaformat mediaformat = Mediaformat.createvideoformat (mime, width, height);   3 null, 0);   4 Mediacodec.start ();  

Here, by giving the decoder a surface, the decoder is able to display the screen directly.

Then the data is processed:

1  Public voidOnframe (byte[] buf,intOffsetintLengthintflag) {  2bytebuffer[] Inputbuffers =mediacodec.getinputbuffers (); 3             intInputbufferindex = Mediacodec.dequeueinputbuffer (-1); 4         if(Inputbufferindex >= 0) {  5Bytebuffer InputBuffer =Inputbuffers[inputbufferindex]; 6 inputbuffer.clear (); 7 inputbuffer.put (buf, offset, length); 8Mediacodec.queueinputbuffer (inputbufferindex, 0, length, MCount * 1000000/frame_rate, 0); 9mcount++; Ten         }   One    AMediacodec.bufferinfo Bufferinfo =NewMediacodec.bufferinfo ();  -        intOutputbufferindex = Mediacodec.dequeueoutputbuffer (bufferinfo,0);  -         while(Outputbufferindex >= 0) {   theMediacodec.releaseoutputbuffer (Outputbufferindex,true);  -Outputbufferindex = Mediacodec.dequeueoutputbuffer (bufferinfo, 0);  -        }   -}

Queueinputbuffer The third parameter is the timestamp, in fact, how to write it does not matter, as long as the linear increase in time can be, here casually get one. The next section of the code is to release the buffer, because we directly let the decoder display, there is no need to decode the data, but must be so released, or the decoder is always left for you, memory is not enough.

Well, by now, it's almost ready. If you're lucky enough, you can see the video now, like on my Samsung phone. However, I have tried several other platforms, most of them can not, there are always a variety of problems, if you want to develop a platform-independent application, there are many problems to solve. Tell me about some of the things I've encountered:

1. Video size

Generally can support 176x144/352x288 this size, but the larger, 640x480 has a lot of machines, as for why, I do not know. Of course, this size must be the same size as the camera preview, and the size of the preview can be enumerated.

2. Color space

According to the Android SDK documentation, make sure that all hardware platforms support colors, the camera preview output is YUV12, and the encoder input is Color_formatyuv420planar, which is set in the previous code. However, the document is ultimately a document, otherwise Android will not be Android.

On some platforms, the two color formats are the same, and the camera's output can be used directly as input to the encoder. There are platforms, two are not the same, the former is YUV12, the latter equals I420, need to reverse the former UV components. The following code is inefficient and can be used for reference.

1 byte[] I420bytes =NULL; 2 Private byte[] swapyv12toi420 (byte[] yv12bytes,intWidthintheight) {  3     if(I420bytes = =NULL)  4I420bytes =New byte[Yv12bytes.length]; 5      for(inti = 0; i < width*height; i++)  6I420bytes[i] =Yv12bytes[i]; 7      for(inti = width*height; I < Width*height + (WIDTH/2*HEIGHT/2); i++)  8I420bytes[i] = yv12bytes[i + (WIDTH/2*HEIGHT/2)]; 9      for(inti = Width*height + (WIDTH/2*HEIGHT/2); I < Width*height + (WIDTH/2*HEIGHT/2); i++)  TenI420bytes[i] = yv12bytes[i-(WIDTH/2*HEIGHT/2)];  One     returni420bytes;  A}

The difficulty here is that I don't know how to tell if I need this conversion. Android 4.3 is said to avoid the problem by taking images from the camera's preview. Here is an example, although I did not read, but looks very strong appearance, should not be wrong (feel Shing). Http://bigflake.com/mediacodec/CameraToMpegTest.java.txt

3. Format of input and output buffers

There is no format in the SDK, but the H264 format is basically appendix B. However, there are more features, it is not with that Startcode, is that 0x000001, make his encoder made out of something to give his decoder, he can not solve his own. Fortunately, we can add it ourselves.

1Bytebuffer OutputBuffer =Outputbuffers[outputbufferindex]; 2 byte[] Outdata =New byte[Bufferinfo.size + 3]; 3Outputbuffer.get (Outdata, 3, bufferinfo.size); 4 if(Framelistener! =NULL) {  5     if((outdata[3]==0 && outdata[4]==0 && outdata[5]==1)  6|| (outdata[3]==0 && outdata[4]==0 && outdata[5]==0 && outdata[6]==1))  7     {  8Framelistener.onframe (Outdata, 3, outdata.length-3, Bufferinfo.flags); 9     }  Ten     Else   One     {   AOutdata[0] = 0;  -OUTDATA[1] = 0;  -OUTDATA[2] = 1;  theFramelistener.onframe (outdata, 0, Outdata.length, bufferinfo.flags);  -     }   -}

4, sometimes die in Dequeueinputbuffer (-1) above

According to the SDK documentation, the parameters of Dequeueinputbuffer indicate the time to wait (in milliseconds), 1 to be equal, and 0 to be unequal. According to common Sense 1 on the line, but actually in a lot of machines will hang off, no way, or pass 0, drop frame is better than hanging off. Of course, you can pass a specific number of milliseconds, but it doesn't mean much.

Android uses MEDIACODEC to implement video hard decoding