PES streams for packaging format parsing of PES,TS,PS,RTP streams

Because of the work exposure to a variety of different audio and video packaging formats, common national standard PS flow, onvif RTP stream and TS flow, and so on, all say good memory than rotten pen, the time to summarize, or in the future can be consulted, because of the level of problems, there may be omissions and problems, please advise one, PES flow

PES flow is the first layer of the original ES stream package, the basic unit of the PES stream is the PES package, composed of Baotou and payload, ES Stream is the audio and video streaming, is the original video audio stream from the encoder, the ES stream contains only one content, which is video or audio, the package does not change it, Only in the front to add the head, such as private head, decoding, the private head stripped, the original ES stream into the decoder decoder decoding, which is also the decoding versatility, if modified, the other decoder can not decode; PEs and ES, are a single source stream, generally I encountered a frame of data placed in a PES package, However, the maximum length of a pes package is 65535 bytes, so one frame of data may be divided into multiple PES; the header format is as follows:

It can be seen that the PES package is composed of fixed Baotou, the optional Baotou and load three parts, wherein the fixed header fixed 6 bytes; PEs packet Length field occupies a bit of 16bit, the maximum value is 65536,

Therefore, a frame may be divided into several PES packages, each of which describes the meaning of each of these fields:

Packet Start code prefix: Baotou starting code, fixed to 0x000001, occupies a bit 24bit;

Stream ID: (UI) The type of load flow in the PES package, the general video is 0XE0, the audio is 0xc0, and the placeholder is 8bit;

pes packetlength: (UI) PES packet lengths, including the length of the optional header and payload after this byte, occupying a bit of 16bit;

Optional PES Header, sequentially:

' 10 ' field : Placeholder 2bit;

PES Scrambling control: Encryption mode, accounting for 2bit;00 unencrypted, 01 or 10 or 11 defined by the user;

PESPriority: Priorities for payloads, placeholder 1bit, high load priority with a value of 1 ratio of 0;

Data Alignment Indicator: Digital positioning indicator, placeholder 1bit;

Copyright Information, 1 for copyright, 0 non-copyright, occupy a bit of 1bit;

Original or copy: original or Backup, 1 for original, 0 for backup, placeholder 1bit;

The following is the 7 flags (typically we are interested in the PTS DTS flag bit):

pts_dts_flags: pts and DTS flag bits, placeholder 2bit;10 indicates that the header has a PTS field, 11 indicates that there are PTS and DTS fields, 00 means none, 01 is forbidden, and no such situation occurs;

escr_flag: ESCR flag, Placeholder 1bit;1 indicates ESCR field first, 0 no this field

es_rate_flag: es_rate field, Placeholder 1bit;1 indicates the header has this field, 0 does not have this field;

dsm_trick_mode_flag: Placeholder 1bit;1 represents a 8-bit dsm_trick_mode_flag field, 0 does not have this field;

additional_copy_info_flag: Placeholder 1bit;1 indicates this field is first, 0 means no this field;

pes_crc_flag: Placeholder 1bit; 1 indicates that the PES group has a CRC field and 0 does not have this field;

pes_extension_flag: Placeholder 1bit; extended flag bit, 1 for extended field, 0 without this field;

pes Header Datalength: (UI) The length of the optional field and fill field in the PES header, placeholder 8bit, the contents of the optional field are controlled by the above 7 flags;

Optional Fields: The description information area of an optional field, whose contents are controlled by the 7 flags above;

Pts/dts field : Display timestamp/decode timestamp, placeholder 40bit, when pts_dts_flags = = 1x This field exists; Time occupies 33 bit,pts and DTS content is in this 40bit to take 33 bits, the same way;

PTS (Presentation time stamp) displays timestamps and DTS (decoding time stamp) decoding timestamps, which are used for audio and video synchronization, in the header of the PES package, and Pts/dts is relative to the SCR (System reference) timestamp, is a 90000-unit, Pts/dts to MS conversion formula is PTS/90, the system clock frequency (H264 sampling frequency?) Is 90Khz, so the conversion to the second is pts/90000, so if the player is in MS, Pts/dts is to use the formula MS=PTS/90 to convert the line, and if the clock frequency is the unit, the Pts/dts will be sent directly into the decoding can, if there is no B-frame, The Order of PTS and DTS should be consistent, if there is a B-frame, you need to decode the P-frame before you can solve the B-frame, so you need pts and DTS to control the decoding time and display time;

byte order in turn:

start_code: Starting code, Placeholder 4bit, if pts_dts_flags = = ' 10 ', then only pts, the starting code is 0010; if pts_dts_flags = = ' 11 ', the PTS and DTS are present, and the starting code for PTS is 0011,dts, which is 0001; (The starting code for PTS is the same as flag for 2 bit)

pts[32..30]: occupies a position of 3bit;

marker_bit: occupies a position of 1bit;

pts[29..15]: occupies a position of 15bit;

marker_bit: occupies a position of 1bit;

pts[14..0]: occupies a position of 15bit;

marker_bit: occupies a position of 1bit;

Pts/dts = (PTS1 & 0x0e) << + (PTS2 & 0xFFFE) << + (PTS3 & 0xFFFE) >> 1;

ESCR field : This field occupies 48bit, consists of 33bit escr_base field and 9bit escr_extension field, Escr_flag = = 1 o'clock this field exists; Data order:

Reserved: reserved field, occupy bit 2bit;

escr_base[32..30]: occupies a position of 3bit;

marker_bit: occupies a position of 1bit;

escr_base[29..15]: occupies a position of 15bit;

marker_bit: occupies a position of 1bit;

escr_base[14..0]: occupies a position of 15bit;

marker_bit: occupies a position of 1bit;

escr_extension: (UI) occupies a bit 9bit; period number, value range 0~299; cycle once, base+1;

marker_bit: occupies a position of 1bit;

ES Rate field : The target decoder receives PES packet byte rates, is forbidden to 0, placeholder 24bit,es_rate_flag = = 1 o'clock this field exists; The data order is:

marker_bit: occupies a position of 1bit;

es_rate: occupies a position of 22bit;

marker_bit: occupies a position of 1bit;

Trick mode control field : Indicates which Trick mode is applied to the corresponding video stream, occupying 8 Bit,dsm_trick_mode_flag = = 1 o'clock this field exists; Trick_mode_ Control occupies the first 3 bits, according to which there are 5 bit different contents after the value;

if Trick_mode_control = = ' 000 ', then the byte order is:

field_id: occupies a position of 2bit;

Intra_slice_refresh: occupies a position of 1bit;

frequency_truncation: occupies a position of 2bit;

if Trick_mode_control = = ' 001 ', then the byte order is:

Rep_cntrl: occupies a position of 5bit;

if Trick_mode_control = = ' 010 ', then the byte order is:

field_id: occupies a position of 2bit;

Reserved: occupies a position of 3bit;

if Trick_mode_control = = ' 011 ', then the byte order is:

field_id: occupies a position of 2bit;

Intra_slice_refresh: occupies a position of 1bit;

frequency_truncation: occupies a position of 2bit;

If trick_mode_control== ' 100 ', then the byte order is:

Rep_cntrl: occupies a position of 5bit;

In other cases, the byte order is:

reserved : Occupies a position of 5bit;

Additional Copy Info field : 8 x Bit,additional_copy_info_flag = = 1 o'clock this field exists; The data order is:

marker_bit: occupies a position of 1bit;

Copy info field : Placeholder 7bit; represents private data related to copyright;

Previous PES CRC field : Placeholder 16bit field containing CRC value, Pes_crc_flag = = 1 o'clock this field exists;

pes extension field : PEs extension field, Pes_extension_flag = = 1 o'clock this field exists; The following is the byte order:

Pes_private_data_flag: Occupy bit 1bit, 1 means there is private data, 0 is none;

pack_header_field_flag: Occupies 1bit, 1 means there is Pack_header_field field, 0 is none;

program_packet_sequence_counter_flag: Placeholder 1bit, 1 indicates there is this field, 0 is none;

p-std_buffer_flag: Placeholder 1bit, set 1 indicates that there is a p-std_buffer field, 0 does not have this field;

reserved Field : 3 bit;

pes_extension_flag_2: Placeholder 1bit, 1 means there is an extension field, 0 does not have this field;

Optional the optional field content order for the field:pes extension field is:

pes_private_data fields : Private data content, placeholder 128bit,pes_private_data_flag = = 1 o'clock this field exists;

pack_header_field field : Pack_header_field_flag = = 1 o'clock This field is present, and the fields are made in the following order:

pack_field_length field : (UI) Specifies the length of the following field, occupying a position of 8bit;

Pack_header_field () : length for pack_field_length designation;

Program_packet_sequence_c