Getting Started with AV data processing: Analysis of video stream in H.

The YUV/RGB handlers and PCM handlers that are described in the first two articles are handlers of the AV raw data. This article begins with a description of the audio stream handler. The program described in this article is a video bitstream handler. The location of the video stream in the video player is shown below.

The program in this article is a code stream parsing program. The program can analyze the basic unit Nalu from the H. A stream, and can simply parse the field of the Nalu header. The program can be modified to achieve different functions of the H-code stream processing.

Principle

The original stream (also known as the "bare stream") is made up of one nalu. Their structure is as shown.

Each of these nalu is separated by a startcode (starting code), and the starting code is divided into two types: 0x000001 (3Byte) or 0x00000001 (4Byte). If the Nalu corresponds to the slice for the beginning of a frame with 0x00000001, otherwise use 0x000001.

The step of parsing the code stream is to search for 0x000001 and 0x00000001 from the stream first, isolate the Nalu, and then analyze the Nalu fields. The procedure of this article realizes the above two steps.

The code throughout the program is in the Simplest_h264_parser () function, as shown below.

/** * Simplest audio-visual data processing example * Simplest mediadata Test * * Lei hua Lei Xiaohua * [email protected] * Communication University/Digital TV Technology * Communication University of China/digital TV technology * http://blog.csdn.net/leixiaohua1020 * * This project contains the following examples of audio and visual testing: * (1) pixel data processing program. Functions that contain RGB and YUV pixel format processing. * (2) Audio sampling data processing program. A function that contains the processing of PCM audio sampling format. * (3) The Code stream analysis program. Nalu can be detached and parsed. * (4) AAC Code Stream analysis program. Adts frames can be detached and parsed. * (5) FLV Encapsulation Format analysis program. You can separate the MP3 audio stream from the FLV. * (6) UDP-RTP Protocol analysis program. You can analyze the udp/rtp/mpeg-ts packets. * * This project contains following samples to handling multimedia data: * (1) Video pixel data handling program. It contains several examples to handle RGB and YUV data. * (2) Audio Sample data Handling program. It contains several examples to handle PCM data. * (3) H. Stream Analysis program. It can parse H. bitstream and analysis Nalu of stream. * (4) AAC Stream analysis program. It can parse AAC bitstream and analysis ADTS frame of stream. * (5) FLV format analysis program. It can analysis FLV file and extract MP3 audio stream. * (6) UDP-RTP Protocol Analysis Program. It can analysis Udp/rtp/mpeg-ts Packet. * */#include <stdio.h> #include <stdlib.h> #include <string.h>typedef enum {nalu_type_slice = 1,nalu_t       YPE_DPA = 2,NALU_TYPE_DPB = 3,NALU_TYPE_DPC = 4,nalu_type_idr = 5,nalu_type_sei = 6,nalu_type_sps     = 7,nalu_type_pps = 8,nalu_type_aud = 9,nalu_type_eoseq = 10,nalu_type_eostream = 11,nalu_type_fill = A,} nalutype;typedef enum {nalu_priority_disposable = 0,nalu_pririty_low = 1,nalu_priority_high = 2,NALU      _priority_highest = 3} nalupriority;typedef struct{int Startcodeprefix_len; //!                 4 for parameter sets and first slice in picture, 3 for everything else (suggested) unsigned len; //!            Length of the NAL unit (excluding the start code, which does not belong to the Nalu) unsigned max_size; //!            Nal Unit Buffer sizeint forbidden_bit; //!        Should is always falseint nal_reference_idc; //! Nalu_priOrity_xxxxint Nal_unit_type; //!                    Nalu_type_xxxx Char *buf; //! Contains the first byte followed by the EBSP} nalu_t;                FILE *h264bitstream = NULL; !< the bit stream fileint info2=0, info3=0;static int FindStartCode2 (unsigned char *buf) {if (buf[0]!=0 | | buf[1]!=0 | | BUF[2]!=1) return 0; 0x000001?else return 1;} static int FindStartCode3 (unsigned char *buf) {if (buf[0]!=0 | | buf[1]!=0 | | BUF[2]!=0 | | BUF[3]!=1) return 0;//0x00000001?else return 1;} int Getannexbnalu (nalu_t *nalu) {int pos = 0;int startcodefound, rewind;unsigned char *buf;if ((Buf = (unsigned char*) call OC (nalu->max_size, sizeof (char))) = = NULL) printf ("Getannexbnalu:could not allocate Buf memory\n"); Nalu->startco Deprefix_len=3;if (3! = Fread (Buf, 1, 3, H264bitstream)) {free (BUF); return 0;} Info2 = FindStartCode2 (Buf), if (Info2! = 1) {if (1! = Fread (buf+3, 1, 1, h264bitstream)) {free (BUF); return 0;} Info3 = FindStartCode3 (Buf), if (Info3! = 1) {free (Buf); return-1;}else {pos = 4;nalu->startcodeprefix_len = 4;}} Else{nalu->startcodeprefix_len = 3;pos = 3;} Startcodefound = 0;info2 = 0;info3 = 0;while (! Startcodefound) {if (feof (H264bitstream)) {Nalu->len = (pos-1)-nalu->startcodeprefix_len;memcpy (Nalu->buf,     &buf[nalu->startcodeprefix_len], Nalu->len); Nalu->forbidden_bit = nalu->buf[0] & 0x80; 1 BITNALU-&GT;NAL_REFERENCE_IDC = nalu->buf[0] & 0x60; 2 Bitnalu->nal_unit_type = (Nalu->buf[0]) & 0x1f;//5 bitfree (BUF); return pos-1;} buf[pos++] = fgetc (h264bitstream); Info3 = FindStartCode3 (&buf[pos-4]); if (Info3! = 1) Info2 = FindStartCode2 (& BUF[POS-3]); Startcodefound = (Info2 = = 1 | | info3 = = 1);}  Here, we had found another start code (and read length of Startcode bytes more than we should//had. Hence, go back in the Filerewind = (Info3 = = 1)? -4: -3;if (0! = fseek (H264bitstream, Rewind, seek_cur)) {free (BUF);p rintf ("Getannexbnalu:cannot fseek in the bit stream File ");} Here the Start code, the complete Nalu, and the next start code are in the Buf. The size of Buf is POS, Pos+rewind is the number of bytes excluding the next//start code, and (Pos+rewind)-STARTCODEP Refix_len is the size of the Nalu excluding the start Codenalu->len = (pos+rewind)-nalu->startcodeprefix_len;memcpy (Nalu->buf, &buf[nalu->startcodeprefix_len], nalu->len);//nalu->forbidden_bit = nalu->buf[0] & 0x80; 1 BITNALU-&GT;NAL_REFERENCE_IDC = nalu->buf[0] & 0x60; 2 Bitnalu->nal_unit_type = (Nalu->buf[0]) & 0x1f;//5 bitfree (BUF); return (pos+rewind);} /** * Analysis H. bitstream * @param URL location of the input H. bitstream file. */int Simplest_h264_parser (char *url) {nalu_t *n;int buffersize=100000;//file *myout=fopen ("Output_log.txt", "wb+"); FILE *myout=stdout;h264bitstream=fopen (URL, "rb+"), if (h264bitstream==null) {printf ("Open file error\n"); return 0;} n = (nalu_t*) calloc (1, sizeof (nalu_t)), if (n = = NULL) {printf ("Alloc Nalu Error\ n "); return 0;} N->max_size=buffersize;n->buf = (char*) calloc (buffersize, sizeof (char)), if (n->buf = = NULL) {free (n);p rintf ( "Allocnalu:n->buf"); return 0;}    int Data_offset=0;int nal_num=0;printf ("-----+--------Nalu Table------+---------+\n");p rintf ("num |    POS |  IDC |   TYPE | LEN |\n ");p rintf ("-----+---------+--------+-------+---------+\n "), while (!feof (H264bitstream)) {int data_lenth; Data_lenth=getannexbnalu (n); char Type_str[20]={0};switch (n->nal_unit_type) {case nalu_type_slice:sprintf (type_ STR, "SLICE"), Break;case nalu_type_dpa:sprintf (type_str, "DPA"), Break;case nalu_type_dpb:sprintf (type_str, "DPB"); Break;case nalu_type_dpc:sprintf (type_str, "DPC"); break;case nalu_type_idr:sprintf (Type_str, "IDR"); Break;case NALU _type_sei:sprintf (Type_str, "SEI"), Break;case nalu_type_sps:sprintf (type_str, "SPS"); Break;case Nalu_type_pps: sprintf (Type_str, "PPS"); break;case nalu_type_aud:sprintf (Type_str, "AUD"); break;case nalu_type_eoseq:sprintf (type _str, "Eoseq"); Break;case nalu_type_eostream:sprintf (Type_str, "Eostream"); break;case nalu_type_fill:sprintf (Type_str, "FILL"); break;} Char Idc_str[20]={0};switch (n->nal_reference_idc>>5) {case nalu_priority_disposable:sprintf (IDC_STR, " Break;case nalu_pririty_low:sprintf (IDC_STR, "low") Break;case nalu_priority_high:sprintf (idc_str, "Dispos"); ); Break;case nalu_priority_highest:sprintf (Idc_str, "highest"); fprintf (Myout, "%5d|%8d|%7s|%6s|%8d|\n", Nal_num,data_offset,idc_str,type_str,n->len);d Ata_offset=data_offset +data_lenth;nal_num++;} Freeif (n) {if (n->buf) {free (n->buf); n->buf=null;} Free (n);} return 0;}

The function call method above is shown below.

Simplest_h264_parser ("sintel.h264");

Results

The input of this program is a file path of the original code stream (bare stream), output as the Nalu statistic of the stream, as shown in.

Download

Simplest mediadata test

Project Home

sourceforge:https://sourceforge.net/projects/simplest-mediadata-test/

Github:https://github.com/leixiaohua1020/simplest_mediadata_test

Open source China: http://git.oschina.net/leixiaohua1020/simplest_mediadata_test

csdn:http://download.csdn.net/detail/leixiaohua1020/9422409

This project contains several examples of audio and visual data parsing:

(1) Pixel data processing program. Functions that contain RGB and YUV pixel format processing.
(2) Audio sampling data processing program. A function that contains the processing of PCM audio sampling format.
(3) The Code Stream analysis program for H. Nalu can be detached and parsed.
(4) AAC Stream Analysis Program. Adts frames can be detached and parsed.
(5) FLV Encapsulation Format parser. You can separate the MP3 audio stream from the FLV.
(6) UDP-RTP Protocol Analysis Program. You can analyze the udp/rtp/mpeg-ts packets.

Rai (Lei Xiaohua)
[Email protected]
http://blog.csdn.net/leixiaohua1020

Getting Started with AV data processing: Analysis of video stream in H.