Android Multimedia Framework Summary (24) Mediamuxer realizes the phone screen recording into a GIF image

Reprint please the head source link and the tail two-dimensional code together reprint, this article from countercurrent fish yuiop:http://blog.csdn.net/hejjunlin/article/details/53866405

Preface: The introduction is to use Mediamuxer and mediaextractor into the audio and video cutting, today with Mediamuxer and Audiorecord and Mediacodec and surface screen recording into GIF. Look at the agenda:

• Main ideas
• Two ways to turn GIF

Mediamuxer is used to mix audio and video to generate multimedia files. The disadvantage is that only one audio track and one video track can be supported at this time, and only MP4 output is supported.

1: Operation steps

2: Note that the GIF is the resulting recording screen after the resulting

Main idea:

Logic: The recorder does not need to manipulate the video raw data, so use inputsurface as input to the encoder.

Video: Mediaprojection the Virtualdisplay incoming surface created by Createvirtualdisplay is returned through the Createinputsurface method of Mediacodec, " This article is from the countercurrent fish yuiop:http://blog.csdn.net/hejjunlin/article/details/53866405 "indicates that the input of the encoder actually comes from the recorded screen data, So we just need to get the encoded bytebuffer in the output buffer of Mediacodec.

Audio: The recording program obtains the audio raw data PCM, passes it to the MEDIACODEC encoding, and then gets the encoded bytebuffer from the MEDIACODEC output buffer.

The audio and video mix is eventually combined with the merge module.

Video: Mediaprojection the Virtualdisplay incoming surface created by Createvirtualdisplay is returned through the Getsurface method of ImageReader, Indicates that the recorded screen frame data is passed to ImageReader, so the relevant API of ImageReader can be read to the recorded screen at each frame of data

Audio: Because the recording is the original PCM encoded audio data, so after recording to the audio data directly call Audiorecord.

It's simply redirecting the direction of the screen-recorded data, what this surface provides, and where the recorded video data goes. Surface provides a local surfaceview control, then the screen content will be displayed on the control, providing MEDIACODEC is the input source as the encoder eventually obtained encoded data, The ImageReader is provided as the ImageReader data source, and the original data stream of the video is finally obtained.

Since the recording is a video, it has to be GIF, there are two options:

• Extract video file--parse video--Extract the Bitmap sequence (use Mediametadataretriever to extract a moment's picture and then encode a number of pictures in a moment into a GIF. It seems to be the principle of GIF, but the implementation of the effect is very poor, can not be accurately extracted to accurate pictures, resulting in the synthesis of GIF graphics can not be coherent playback, play up also jump frame jump very badly. Can be described by the appalling.)
• The use of FFmpeg direct to GIF, this in my "FFmpeg under Linux Install the compilation process" in the article, is compiled out of the library, the demo to GIF, then is the Superindicator gif. For putting on Android, it is the same principle. This method of post-hillock.

Solution of a thought video file analysis

After the video file is read successfully, the next step is to parse the video file, select the video clips that need to be converted, and extract the Bitmap sequence. The following is a concrete implementation, extraction BITMAP sequence is based on the given start time and end time and frame rate from the video file to obtain the corresponding Bitmap, this article is mainly using the mediametadataretriever provided by the API to achieve, before looking at the code can look at the first Mediametadataretriever API documentation, the core function of this class is to get the video frame and metadata, the following is the core implementation code:

 PublicList<bitmap>Createbitmaps(String Path) {Mediametadataretriever MMR =NewMediametadataretriever (); Mmr.setdatasource (path);Doubleinc = +* +/fps; for(Doublei = begin; I < end; i + = Inc) {Bitmap frame = Mmr.getframeattime (Long) I, mediametadataretriever.option_closest);if(Frame! =NULL) {Bitmaps.add (frame); }  }returnbitmaps;}PrivateBitmap Scale(Bitmap Bitmap) {returnBitmap.createscaledbitmap (Bitmap, Width >0? Width:bitmap.getWidth (), Height >0? Height:bitmap.getHeight (),true);}

To get the Bitmap sequence to generate the GIF, the next thing to do is to encode the data in the Bitmap sequence in the GIF file format to produce the final GIF file. The goal is clear, and the next step is to see the concrete implementation process.

Introduction to GIF Format

GIF file before the need to introduce the storage format of the GIF, GIF format related articles More, there is no need to introduce too detailed, but simply say later procedures will be used in the aspects.
A GIF image is based on a color list (the stored data is the index value of the point's color corresponding to the color list) and supports up to 8 bits (256 colors). GIF files are divided into many memory blocks, which are used to store multiple images or control blocks that determine the behavior of the image, for animating and interactive applications. GIF files also compress image data by LZW compression algorithms to reduce image size.
GIF files are built into blocks, including both control blocks and data blocks. Control block is to control the behavior of the block, depending on the control block contains a number of different control parameters, the data block contains only some 8-bit character stream, the control block in front of it to determine its function, each data block 0 to 255 bytes, the first byte of the data block indicates the size of the block (bytes), This byte is not included when calculating the size of the block, so an empty chunk has one byte, which is the size of the data block 0x00.

GIF File Write

Just start to contact GIF file will feel more complex, storage format, encoding format and so more complex than Bitmap, but can actually simplify the understanding of the problem, generate GIF and generate Bitmap principle is similar, is in accordance with the prescribed format to write the file on the line, not too tangled inside details, Otherwise, you will be caught in the tedious details (commonly known as a dead end) and neglect the ultimate goal just to generate GIF files. Here's a look at some of the file parts that need to be written:

Extracting pixel values from Bitmap
First need to get the above Bitmap pixel values extracted, convenient to write the pixel values in the GIF file, while extracting the pixel value, the creation of the GIF file required color table, the generation of color table process is more complex, here do not post source code, interested can Google color quantization algorithm, Not interested directly with ready-made, the following is the specific implementation of extracting pixel values:

protected void Getimagepixels() {intW = image.getwidth ();inth = image.getheight (); pixels =New byte[w*h*3]; for(inti =0; I < H; i++) {intStride = W *3I for(intj =0; J < W; J + +) {intp = Image.getpixel (J, I);intStep = J *3;intOffset = stride + step;//Bluepixels[offset+0] = (byte) (P &0x0000FF) >>0);//Greenpixels[offset+1] = (byte) (P &0x00FF00) >>8);//Redpixels[offset+2] = (byte) (P &0xFF0000) >> -); } }}

GIF file Headers (header)
The header section totals 6 bytes, including: GIF signature and version number, GIF signature consists of 3 characters "GIF", a total of 3 bytes, the version number is also composed of 3 bytes, can be "87a" or "89a" (respectively, 1987 and 1989 versions), the implementation code is as follows:

// 写入文件头protectedvoidwriteHeaderthrows IOException { writeString("GIF89a");}protectedvoidwriteStringthrowsfor (int0; i < s.length(); i++) { out.write((byte) s.charAt(i)); }}

Logical screen Identifier (Logical screens descriptor)
The file header is followed by a logical screen identifier (Logical descriptor), which consists of 7 bytes, which defines the size of the GIF image, the color depth, the background color, and the number of indexes with no global color list and color list. The implementation code is as follows:

Write Logical screen Identifiers

protected void WRITELSD() throws IOException {writeshort (width);//write image widthWriteshort (height);//write image Height  out. Write (0x80|//Global Color list flag 1    0x70|//Determine the color depth of the image (7+1=8)    0x00|//Global Color list sorted by 0    0x07));Number of indexes (7+1 of 2) for the//color list  out. Write (0);//Background color (index in global Color list)  out. Write (0);//pixel aspect ratio default 1:1}protected void Writeshort(int value) throws IOException { out. Write (value&0xFF); out. Write (value>>8) &0xFF);}

Logical screen identifier part of the structure is slightly more complex, if you do not know what the meaning of each person can refer to: GIF graphics file format document in the logical screen identifier section.
Global color Table
The global color list must immediately follow the logical screen identifier, each color list index entry consists of three bytes, according to the order of R, G, B, the implementation of the specific generation of color table can be seen in the source section, due to the complexity of the generation process, "this article from the counter-current fish yuiop:http:// blog.csdn.net/hejjunlin/article/details/53866405 "Here is not the code generated by the color table, the following is the code to write the color table:

// 写入颜色表protectedvoidwritePaletteout0int n = (3256for (int0out.write(0); }

Graphical control Extensions (Graphic-Extension)
This section is optional, supported by the 89a version, and can be placed in front of a block of images (including image identifiers, local color lists, and image data) or text extension blocks to control the rendering (render) Form of the first image (or text) following it, implementing the code below:

protected void Writegraphicctrlext() throws IOException { out. Write (0x21);//extension block ID, fixed value 0x21  out. Write (0xf9);//Graphics Control Extension tab, fixed value 0xf9  out. Write (4);//block size, fixed value 4  out. Write (0|//1:3 reserved bit   0|//4:6 do not use disposal method   0|//7 user input flag 0   0);//8 Transparent color marker 0Writeshort (delay);//Delay Time  out. Write (0);//Transparent color index value  out. Write (0);//block terminator, fixed value 0}

Image identifier (image descriptor)
A GIF file can contain multiple images, an image is immediately followed by an image identifier, the image identifier begins with the 0x2C (', ') character, and defines the nature of the image immediately following it, including the offset of the image relative to the logical screen boundary, Image size and whether there are local color lists and color list sizes, consisting of 10 bytes, the following is the implementation code:

protected   void  writeimagedesc  () throws IOException {out . Write (0x2c );  //image identifier starts with a fixed value of 0x2c  writeshort (0 );  //x-Direction offset  writeshort (0 ); //y-direction offset  writeshort (width); //image width  writeshort (height); //Image height   out . Write ((0x80  |  //local color list flag 1   0x00  |  0x00  |  0x07 )); //the number of indexes of the local color list (2 of 7+1 Times) }  

Images (image data)
GIF image data using the LZW compression algorithm, greatly reducing the size of the image data, the specific LZW compression algorithm can be Google, the implementation of the program can refer to the bottom of the article source link. The following is a write implementation of the image data:

protectedvoidwritePixelsnew LZWEncoder(  width, height, indexedPixels, colorDepth); encoder.encode(out);}

File Terminator (Trailer)
This section has only one byte, which identifies a GIF file ending with a fixed value of 0x3B, implementing the Code:

publicvoidfinishout.write(0x3boutout.close();}

Summarize
So far, the implementation of converting MP4 files to GIF files has basically been done, if you need to clip gif files, add watermarks, etc., you can write to GIF Bitmap sequence before writing to "this article from the Countercurrent fish yuiop:http://blog.csdn.net /hejjunlin/article/details/53866405 "Bitmap for the corresponding treatment, if there is any problem welcome to Exchange learning. I hope the content of this article will be helpful to everyone's study work.

Scenario Two ideas:

To compile the so file procedure:

Compiling the resulting so will automatically generate a libs directory:

Click into the Libs directory to find a so folder for the arm platform, and a so folder for the x86 platform:

Just click on one, enter, is some so:

Experience APK

: Link: Https://pan.baidu.com/s/1skR35nB Password: 2wb3

The first time to get blog update reminders, as well as more Android dry, source code Analysis , Welcome to follow my public number, sweep the bottom QR code or long press to identify two-dimensional code, you can pay attention to.

If you feel good, easy to praise, but also to the author's affirmation, can also share this public number to you more people, original not easy

Data reference: (http://www.jb51.net/article/91305.htm)

Android Multimedia Framework Summary (24) Mediamuxer realizes the phone screen recording into a GIF image