Key hardware decoding and decryption and decoding Technologies

Key hardware decoding and decryption and decoding Technologies

In the previous article, we used ffmpeg to separate audio and video data in a multimedia container, but it is likely that the data cannot be correctly decoded. Why? Before decoding the data, you need to configure the decoder. A typical example is the combination of H264 + AAC, a popular "golden partner. This article describes how to parse key decoding configuration parameters of H264 and AAC. Without such configuration information, data frames are often incomplete, resulting in decoder failure to decode.

• Analysis of H264 configuration information

  We know that ffmpeg's avformat_find_stream_info function can obtain multiple types of audio and video media, such as playback duration, audio and video compression format, audio track information, subtitle information, frame rate, sampling rate, and so on. There is an extended data description in the information result (avcodec. h file ):

AVCodecContext is defined as follows:

If the video stream is H264, this extradate contains the configuration information of H264, which is defined as follows:

For more information, see the ISO-14496-15 AVC file format document. The most important information is the NAL length and SPS, PPS data and the corresponding length information. The existing function ff_h1__decode_extradata is used to parse the data in ffmpeg. In my project, the extended data is parsed by myself.

• Parsing and setting AAC configuration information

  If the Audio Data is an AAC Stream, the ADTS (Audio Data Transport Stream) header is required during decoding. This header is not available in container encapsulation or streaming media. Many of my friends do not play audio during AAC stream playback, which is probably caused by this.

  The data required by ADTS is still stored in the above extended data extradata. we need to decode the extended data first, and then re-encapsulate it into the ADTS header information from the decoded data information, the decoder is sent before each frame of AAC data is added, so that the decoder can be decoded normally.

  The extradate data is defined as follows:

   

  For more information, see AudioSpecificConfig in ISO-IEC-14496-3 (Audio. The most important parts are the sampling frequency, Channel configuration, and audio object type. These are generally the configuration parameters required by the AAC decoder.

  This data also has the corresponding decoding function in ffmpeg: avpriv_aac_parse_header. In my project, I did not use this function, but implemented it myself:

• typedef struct{      int write_adts;      int objecttype;      int sample_rate_index;      int channel_conf;}ADTSContext;

  　　

  int aac_decode_extradata(ADTSContext *adts, unsigned char *pbuf, int bufsize){      int aot, aotext, samfreindex;      int i, channelconfig;      unsigned char *p = pbuf;         if (!adts || !pbuf || bufsize<2)      {            return -1;      }      aot = (p[0]>>3)&0x1f;      if (aot == 31)      {            aotext = (p[0]<<3 | (p[1]>>5)) & 0x3f;            aot = 32 + aotext;            samfreindex = (p[1]>>1) & 0x0f;                         if (samfreindex == 0x0f)            {                  channelconfig = ((p[4]<<3) | (p[5]>>5)) & 0x0f;            }            else            {                  channelconfig = ((p[1]<<3)|(p[2]>>5)) & 0x0f;            }      }      else      {            samfreindex = ((p[0]<<1)|p[1]>>7) & 0x0f;            if (samfreindex == 0x0f)            {                  channelconfig = (p[4]>>3) & 0x0f;            }            else            {                  channelconfig = (p[1]>>3) & 0x0f;            }      }   #ifdef AOT_PROFILE_CTRL      if (aot < 2) aot = 2;#endif      adts->objecttype = aot-1;      adts->sample_rate_index = samfreindex;      adts->channel_conf = channelconfig;      adts->write_adts = 1;         return 0;}

  　　

  The above pbuf is extradata.

  Next, use ADTSContext data encoding to insert the information in the ADTS header before each AAC frame:

• int aac_set_adts_head(ADTSContext *acfg, unsigned char *buf, int size){             unsigned char byte;         if (size < ADTS_HEADER_SIZE)      {            return -1;      }             buf[0] = 0xff;      buf[1] = 0xf1;      byte = 0;      byte |= (acfg->objecttype & 0x03) << 6;      byte |= (acfg->sample_rate_index & 0x0f) << 2;      byte |= (acfg->channel_conf & 0x07) >> 2;      buf[2] = byte;      byte = 0;      byte |= (acfg->channel_conf & 0x07) << 6;      byte |= (ADTS_HEADER_SIZE + size) >> 11;      buf[3] = byte;      byte = 0;      byte |= (ADTS_HEADER_SIZE + size) >> 3;      buf[4] = byte;      byte = 0;      byte |= ((ADTS_HEADER_SIZE + size) & 0x7) << 5;      byte |= (0x7ff >> 6) & 0x1f;      buf[5] = byte;      byte = 0;      byte |= (0x7ff & 0x3f) << 2;      buf[6] = byte;         return 0;}

  　　

  This header is a fixed length of 7 bytes, so the seven bytes can be reserved in advance for ADTS.

  Through the above extended data processing of H264 and AAC, it should be okay to play the multimedia files, streaming media, and on-demand video of various "golden partners.

  To get more original articles in the first place, please follow the personal public platform: coder_online, scan the QR code below or search for coder_online. There are a large number of Android, Chromium, linux and other related articles are waiting for you. We can also communicate online.

  From: http://my.oschina.net/u/2336532/blog/400790