Key extension data processing for decrypting H264 and AAC hardware decoding

In the previous article, we used ffmpeg to separate the audio and video data from a multimedia container, but it is possible that the data could not be decoded correctly. Why is it? Because the decoder needs to be configured before decoding the data, it is typical of the current popular HD coded "Golden Partner" combination H264 + AAC . This article will describe the key decoding configuration parameters of H264 and AAC, and without these configuration information, the data frames are often incomplete, causing the decoder to not decode.

• configuration information parsing for H264

  As we know earlier, FFmpeg 's avformat_find_stream_info function can obtain many kinds of audio and video media, such as playback duration, audio and video compression format, audio track information, subtitle information, frame rate, sampling rate and so on. In the information results there is an extended data description (in the avcodec.h file):

Avcodeccontext is defined as follows:

If the video stream is H264, the Extradate contains H264 configuration information, which is defined as follows:

You can refer to the "iso-14496-15 AVC file format" document for detailed explanations . The most important is the nal length and SPS, PPS data and the corresponding length information. The parsing of this data has a ready-made function in FFmpeg: Ff_h264_decode_extradata, in my project, I write the extended data parsing.

• AAC configuration information parsing and setting

  stream, need Adts (Audio Data Transport stream) head, whether it's container encapsulation or streaming media, without this, it's generally not playable. Many friends are doing aac

  ADTS The data required is still placed in the above extension data extradata , we need to decode the extension data, and then re-encapsulated from the decoded data information into ADTS header information, add to each frame AAC Before the data is sent to the decoder, so it can be decoded normally.

  The extradate data is defined as follows:

  For more information and instructions, please refer to the Audiospecificconfig section of "Iso-iec-14496-3 (Audio)" . The most important part of this is the sampling frequency, channel configuration, and audio object type, which are generally the configuration parameters required by the AAC decoder.

  This data also has the corresponding decoding function in the ffmpeg: Avpriv_aac_parse_header. In my project, I did not use this function, but I implemented it myself:

typedef struct{INT Write_adts;   int objecttype;   int sample_rate_index; int channel_conf;} Adtscontext;

Int aac_decode_extradata (adtscontext *adts, unsigned char *pbuf, int  BufSize) {      int aot, aotext, samfreindex;       int i, channelconfig;      unsigned char * p = pbuf;       if  (!adts | |  !pbuf | | &NBSP;BUFSIZE&LT;2)       {             return -1;      }      aot  =  (p[0]>>3) &0x1f;      if  (aot == 31)        {            aotext  =  (p[0]<<3 |  (p[1]>>5))  & 0x3f;             aot = 32 + aotext;             samfreindex =  (p[1]>>1)  & 0x0f;                        if  (samfreindex == 0x0f)             {                    channelconfig =  (p[4]<<3  |  (p[5]>>5))  & 0x0f;             }             else            {                    channelconfig =  ((P[1]&LT;&LT;3) | (p[2]>>5))  & 0x0f;            }       }      else      {             samfreindex =  ((p[0]<<1) |p[1 ]&GT;&GT;7)  & 0x0f;            if   (samfreindex == 0x0f)             {                    channelconfig =  (p[4]>>3)  & 0x0f;             }            else             {                   channelconfig =  (p[1]>>3)  & 0x0f;            }       }  #ifdef  AOT_PROFILE_CTRL      if  (AOT &NBSP;&LT;&NBSP;2)  aot = 2; #endif       adts->objecttype =  aot-1;      adts->sample_rate_index = samfreindex;       adts->channel_conf = channelconfig;       adts->write_adts = 1;       return 0;}

the pbuf above is Extradata.

Next, use the Adtscontext data encoding to insert the Adts header information in front of each AAC frame:

Int aac_set_adts_head (adtscontext *acfg, unsigned char *buf, int size) {              unsigned char byte;        if  (size < adts_header_size)        {            return -1; &NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;}&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;BUF[0]  = 0xff;      buf[1] = 0xf1;       byte = 0;      byte |=  (acfg->objecttype &  0x03)  << 6;      byte |=  (acfg->sample_rate_ index & 0x0f)  << 2;      byte |=  (acfg- >channel_conf &&nbSP;0X07)  >> 2;      buf[2] = byte;       byte = 0;      byte |=  (acfg->channel_ conf & 0x07)  << 6;      byte |=  (adts_ Header_size + size)  >> 11;      buf[3] =  byte;      byte = 0;      byte |=   (adts_header_size + size)  >> 3;      buf[4]  = byte;      byte = 0;       byte |=  ((adts_header_size + size)  & 0x7)  << 5;       byte |=  (0x7ff >> 6)  & 0x1f;       buf[5]&nbSp;= byte;      byte = 0;      byte  |=  (0x7ff & 0x3f)  << 2;      buf[6]  = byte;       return 0;}

The head is a fixed 7- byte length, so the 7 bytes can be vacated in advance for Adts occupancy.

Through the above to H264 and AAC Extended data processing, play all kinds of "gold partner" multimedia files, streaming media, video-on-demand, etc. should be no problem.

Want to get more original articles in the first time, please pay attention to the personal public platform: Programmer Interaction Alliance (coder_online), sweep the QR code below or search number Coder_online can pay attention to, there is a lot of Android, Chromium , Linux and Other related articles waiting for you, we can also communicate online.

If you want to reprint this article, please specify the source: Thank you for your cooperation!

Key extension data processing for decrypting H264 and AAC hardware decoding