Android Audio Video In depth five perfect recording video (with source download)

This project address, the name is recorded video, for star

Https://github.com/979451341/Audio-and-video-learning-materials

This time the code record video can be used in each player, sometimes long display, compared to the previous code to say why the two effects are different, but I first add some of the previously missing Mediacodec official notes and Mediacodec.bufferinfo

1.MediaCodec Supplement

BUFFER_FLAG_CODEC_CONFIG: Hint flags and other specific data containing the encoding initialization/codec, rather than the media data buffer.

Buffer_flag_end_of_stream: The end of this signal stream

Buffer_flag_sync_frame: The synchronized frame buffer that contains the data.

Info_output_buffers_changed: The output buffer has changed and the customer must return Getoutputbuffers () to the output buffer newly set.

Info_output_format_changed: The output format has changed and subsequent data will follow the new format.

Info_try_again_later: Indicates that the call timed out, timeout call Dequeueoutputbuffer

Dequeueinputbuffer (Long Timeoutus): Returns the index of the input buffer to fill valid data or-returns 1 if no such buffer is currently present.

Dequeueoutputbuffer (mediacodec.bufferinfo info, long timeoutus): Will output buffers, blocking the timeoutus subtle

Flush (): Refreshes the input and output port components, all indicators previously returned call Dequeueinputbuffer (long) and Dequeueoutputbuffer (Mediacodec.bufferinfo, long) are invalid.

Mediacodecinfo getcodecinfo (): Gets the codec information.

Getinputbuffers (): This start () is returned after the call.

Getoutputbuffers (): Dequeueoutputbuffer signal output buffer changes when start () is returned info_output_buffers_changed

Mediaformat GetOutputFormat (): This is called Dequeueoutputbuffer signal returns info_output_format_changed format changes

Queueinputbuffer (int index, int offset, int size, long presentationtimeus, int flags): After an input buffer is populated on the specified index, it is submitted to the component.

Mediacodec.bufferinfo Each buffer metadata includes an offset and size that specifies the range of valid data in the associated codec buffer. I understand that there are some tuning parameters that need to be made to write the buffer data locally.

2. Code Comparison
No nonsense directly look at the video encoding This section, first compare Mediaformat

    mediaFormat = MediaFormat.createVideoFormat(MIME_TYPE, this.mWidth, this.mHeight);    mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, BIT_RATE);    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, FRAME_RATE);    mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);    mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, IFRAME_INTERVAL);    final MediaFormat format = MediaFormat.createVideoFormat(MIME_TYPE, mWidth, mHeight);    format.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);  // API >= 18    format.setInteger(MediaFormat.KEY_BIT_RATE, calcBitRate());    format.setInteger(MediaFormat.KEY_FRAME_RATE, FRAME_RATE);    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 10);

What are the width and height of the video format are the same, even the frame rate, acquisition point size are the same, only a different mediaformat.key_color_format, this official note is:
The encoder is set by the user and is readable in the output format of the decoder.

That is, it can set the encoder, can set the encoding method, that is, the two projects the biggest difference is the code, we continue to compare

private void Encodeframe (byte[] input) {
LOG.W (TAG, "Videoencoderthread.encodeframe ()");

    // 将原始的N21数据转为I420    NV21toI420SemiPlanar(input, mFrameData, this.mWidth, this.mHeight);    ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();    ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();    int inputBufferIndex = mMediaCodec.dequeueInputBuffer(TIMEOUT_USEC);    if (inputBufferIndex >= 0) {        ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];        inputBuffer.clear();        inputBuffer.put(mFrameData);        mMediaCodec.queueInputBuffer(inputBufferIndex, 0, mFrameData.length, System.nanoTime() / 1000, 0);    } else {        Log.e(TAG, "input buffer not available");    }

—————-omitted
}

  protected void encode (final bytebuffer buffer, final int length, final long Presentationtimeus) {if (!misca    pturing) return;    Final bytebuffer[] inputbuffers = Mmediacodec.getinputbuffers ();        while (miscapturing) {final int inputbufferindex = Mmediacodec.dequeueinputbuffer (timeout_usec);            if (inputbufferindex >= 0) {final Bytebuffer inputbuffer = Inputbuffers[inputbufferindex];            Inputbuffer.clear ();            if (buffer! = null) {inputbuffer.put (buffer); }

if (DEBUG) log.v (TAG, "Encode:queueinputbuffer");
if (length <= 0) {
Send EOS
Miseos = true;
if (DEBUG) log.i (TAG, "Send Buffer_flag_end_of_stream");
Mmediacodec.queueinputbuffer (inputbufferindex, 0, 0,
Presentationtimeus, Mediacodec.buffer_flag_end_of_stream);
Break
} else {
Mmediacodec.queueinputbuffer (inputbufferindex, 0, length,
Presentationtimeus, 0);
}
Break
} else if (Inputbufferindex = = Mediacodec.info_try_again_later) {
Wait for MEDIACODEC encoder are ready to encode
Because Mediacodec#dequeueinputbuffer (timeout_usec)
Would wait for maximum timeout_usec (10msec
}
}
}

, the truth is no different, there is only one difference that is the first encoding function at the beginning of the use of this, and then at the time of encoding using the I420bytes, the i420bytes by Xnv21bytes transformation

private static void NV21toI420SemiPlanar(byte[] nv21bytes, byte[] i420bytes, int width, int height) {    System.arraycopy(nv21bytes, 0, i420bytes, 0, width * height);    for (int i = width * height; i < nv21bytes.length; i += 2) {        i420bytes[i] = nv21bytes[i + 1];        i420bytes[i + 1] = nv21bytes[i];    }}

We look at each time coding a frame into the mixer is what I do, the following code means to listen to the encoding of a frame before and after the state of the encoder changes and put the encoded data into the MP4 file, and then free memory

            drain();            // request stop recording            signalEndOfInputStream();            // process output data again for EOS signale            drain();            // release all related objects            release();

We're looking at what drain () says,
At the beginning of the Mmediacodec.getoutputbuffers output data, and then get the state of the encoder, if timed out to exit the current loop, if the output buffer has changed, it is executed once mmediacodec.getoutputbuffers, if the output format changed Encoder configuration Mediaformat, and then the encoder again added to the mixer, the status of the value of less than 0 is unpredictable state, since it is not predictable, there is no way, the rest is the normal state, with the Bufferinfo to write data to the mixer

protected void drain() {    if (mMediaCodec == null) return;    ByteBuffer[] encoderOutputBuffers = mMediaCodec.getOutputBuffers();    int encoderStatus, count = 0;    final MediaMuxerWrapper muxer = mWeakMuxer.get();    if (muxer == null) {

throw new NullPointerException ("Muxer is unexpectedly null");
LOG.W (TAG, "muxer is unexpectedly null");
Return
}
Loop:while (miscapturing) {
Get encoded data with maximum timeout duration of timeout_usec (=10[msec])
Encoderstatus = Mmediacodec.dequeueoutputbuffer (Mbufferinfo, timeout_usec);
if (Encoderstatus = = Mediacodec.info_try_again_later) {
Wait 5 counts (=timeout_usec x 5 = 50msec) until Data/eos come
if (!miseos) {
if (++count > 5)
Break LOOP; Out of While
}
} else if (encoderstatus = = mediacodec.info_output_buffers_changed) {
if (DEBUG) log.v (TAG, "info_output_buffers_changed");
This shoud isn't come when encoding
Encoderoutputbuffers = Mmediacodec.getoutputbuffers ();
} else if (encoderstatus = = mediacodec.info_output_format_changed) {
if (DEBUG) log.v (TAG, "info_output_format_changed");
This status indicate the output format of codec is changed
This should come only once before actual encoded data
But this status never come in Android4.3 or less
And in the case, the should treat when mediacodec.buffer_flag_codec_config come.
if (mmuxerstarted) {//second time request is error
throw new RuntimeException ("format changed twice");
}
Get output format from codec and pass them to Muxer
GetOutputFormat should is called after info_output_format_changed otherwise crash.
Final Mediaformat format = Mmediacodec.getoutputformat (); API >= 16
Mtrackindex = muxer.addtrack (format);
Mmuxerstarted = true;
if (!muxer.start ()) {
We should wait until Muxer is ready
Synchronized (muxer) {
while (!muxer.isstarted ())
try {
Muxer.wait (100);
} catch (Final Interruptedexception e) {
Break LOOP;
}
}
}
} else if (Encoderstatus < 0) {
Unexpected status
if (DEBUG) LOG.W (TAG, "drain:unexpected result from Encoder#dequeueoutputbuffer:" + encoderstatus);
} else {
Final Bytebuffer encodeddata = Encoderoutputbuffers[encoderstatus];
if (Encodeddata = = null) {
This never should Come...may is a MEDIACODEC internal error
throw new RuntimeException ("Encoderoutputbuffer" + encoderstatus + "was null");
}
if ((Mbufferinfo.flags & mediacodec.buffer_flag_codec_config)! = 0) {
You shoud set output format to muxer this when you target Android4.3 or less
But Mediacodec#getoutputformat can don't call here (because info_output_format_changed don ' t come yet)
Therefor we should expand and prepare output format from buffer data.
This is the for api>=18 (>=android 4.3), just ignore this flag here
if (DEBUG) log.d (TAG, "drain:buffer_flag_codec_config");
mbufferinfo.size = 0;
}

  if (mbufferinfo.size! = 0) {//encoded data is ready, clear waiting counter                Count = 0;                    if (!mmuxerstarted) {//Muxer is not ready...this would prrograming failure.                throw new RuntimeException ("Drain:muxer hasn ' t started");                }//write encoded data to muxer (need to adjust presentationtimeus.                Mbufferinfo.presentationtimeus = Getptsus ();                Muxer.writesampledata (Mtrackindex, Encodeddata, Mbufferinfo);            Prevoutputptsus = Mbufferinfo.presentationtimeus;            }//return buffer to encoder Mmediacodec.releaseoutputbuffer (Encoderstatus, false);                if (Mbufferinfo.flags & mediacodec.buffer_flag_end_of_stream)! = 0) {//when EOS come.                Miscapturing = false;      Break Out of a while}}}}  

After finishing, the code is many, a lot of abstract understanding, heavy in the process of understanding, details ..... , self-summary

Android Audio Video In depth five perfect recording video (with source download)