Code implementation for PS encapsulation of h264 code streams

1. Before you start:

Recently, because of the new work to maintain the GB module code left by others, I first became familiar with the process, and then tried to encapsulate the PS stream. The results can also pass the test to preview normally. Of course, the headache of developing and reading documents is that you may encounter issues such as Preview screens and card frames. At that time, you gradually read the documents and sort out the logic, below we will share some general process code. Since it is connected to the national standard, it is essential to read its standard documents and related RFC documents! I will not talk about the specifics. You can use Baidu and Google.

Note:Because GB requires PS encapsulation and the RTP Header Format, I also add the RTP Header in the following code. If you do not need it, simply block the RTP in the code.

2. encapsulation focus

When we obtain a frame of audio/video data from the read buffer, each frame actually has only one PS header and a PSM header. If it is an I frame, one more system header, one or more PES headers, and RTP headers,

For example, if the frame data is too long, you have to perform the sharding. Each slice will contain an PES header. The best RTP Load Length is 1460, so we will perform the subcontracting operation! Therefore, each packet has at least one RTP + databuf, and each piece of data has at least one RTP + PES + databuf, each frame must contain at least RTP + PS + PSM + PES + databuf (for key frames, an additional system header)

3. Specific Code implementation for each Encapsulation

First, give a whole EncapsulationRTP-> PS-> sys-> PSM-> PES (if only PS is used, PS-> sys-> SMS-> PES)General process,

Then, we will list the encapsulated interfaces of each component one by one.

/***** @ Remark: Package audio and video data into a PS stream and encapsulate it into RTP * @ Param: pdata [in] the audio and video data to be sent * nframelen [in] the length of the sent data * ppacker [in] packet information, including the time stamp, RTP data buff, socket information sent * stream_type [in] Data Type 0 Video 1 audio * @ return: 0 success others failed */INT gb281820.streampackageforh264 (char * pdata, int nframelen, data_info_s * ppacker, int stream_type) {char sztemppackethead [256]; int nsizepos = 0; int nsize = 0; char * pbuff = NULL; memset (SZ Temppackethead, 0,256); // 1 package for PS header gb28181_make_ps_header (sztemppackethead + nsizepos, ppacker-> s64curpts); nsizepos + = ps_hdr_len; // 2 system header if (ppacker-> IFRAME = 1) {// if it is an I frame, add the system header gb28181_make_sys_header (sztemppackethead + nsizepos); nsizepos + = sys_hdr_len; // map is added to both I and P frames. It seems that I frame addition is not a problem. // gb28181_make_psm_header (sztemppackethead + nsizepos ); // nsizepos + = psm_hd R_len;} // The PSM header (also map) gb28181_make_psm_header (sztemppackethead + nsizepos); nsizepos + = psm_hdr_len; // Add RTP to send it out. In this case, only the PES header and RTP Header are added to the data in the backend. If (gb28181_send_rtp_pack (sztemppackethead, nsizepos, 0, ppacker )! = 0) Return-1; // move backward here to facilitate the copying of PES headers // This is to reduce the waste of space for copying large volumes of raw audio and video data. So move backward here, during actual processing, pay attention to the problem of cross-border address and overwrite pbuff = pdata-pes_hdr_len; while (nframelen> 0) {// The length of each frame cannot exceed the short type, after that, the score piece is sent into the loop nsize = (nframelen> ps_pes_payload_size )? Ps_pes_payload_size: nframelen; // Add the PES header gb28181_make_pes_header (pbuff, stream_type? 0xc0: 0xe0, nsize, (ppacker-> s64curpts/100), (ppacker-> s64curpts/300 )); // Add the RTP Header and send the data if (gb28181_send_rtp_pack (pbuff, nsize + pes_hdr_len, (nsize = nframelen )? 1:0), ppacker )! = 0) {printf ("gb281820.send_pack failed! \ N "); Return-1 ;}/// the data movement pointer operation nframelen-= nsize each time a shard is sent; // only nsize is moved here, because the length of the PES header is moved backward while, pbuff + = nsize;} return 0 ;}

The entire Packaging and Packaging Process is listed above. Next, let's look at the interface one by one.

/***** @ Remark: the encapsulation of the PS header. The specific data in it has been filled in. Refer to the standard * @ Param: pdata [in] fill in the PS header data address * s64src [in] timestamp * @ return: 0 success, others failed */INT gb28181_make_ps_header (char * pdata, unsigned long s64scr) {unsigned long lscrext = (s64scr) % 100; s64scr = s64scr/100; // This is because the video frequency returned by the SDP protocol is 100, the frame rate is 25 frames/s, so the increasing volume is 3600 each time, // you should actually process the timestamp according to your own encoding to ensure that the increment of the timestamp is 3600. // if this is not the case, bits_buffer_s bitsbuffer; bitsbuffer. I _size = ps_hdr_len; bitsbuffer. I _data = 0; bitsbuffer. I _mask = 0x80; // binary: 10000000 This is used for subsequent operations on each bit of a byte to prevent the size of the byte from disordered bitsbuffer. p_data = (unsigned char *) (pdata); memset (bitsbuffer. p_data, 0, ps_hdr_len); bits_write (& bitsbuffer, 32, 0x000001ba);/* Start codes */bits_write (& bitsbuffer, 2, 1 ); /* marker bits '01b '*/bits_write (& bitsbuffer, 3, (s64scr> 30) & 0x07);/* system clock [32 .. 30] */bits_write (& bitsbuffer, 1, 1);/* marker bit */bits_write (& bitsbuffer, 15, (s64scr> 15) & 0x7fff ); /* system clock [29 .. 15] */bits_write (& bitsbuffer, 1, 1);/* marker bit */bits_write (& bitsbuffer, 15, s64scr & 0x7fff);/* system clock [29 .. 15] */bits_write (& bitsbuffer, 1, 1);/* marker bit */bits_write (& bitsbuffer, 9, lscrext & 0x01ff);/* system clock [14 .. 0] */bits_write (& bitsbuffer, 1, 1);/* marker bit */bits_write (& bitsbuffer, 22, (255) & 0x3fffff ); /* bit rate (N Units of 50 bytes per second .) */bits_write (& bitsbuffer, 2, 3);/* marker bits '11' */bits_write (& bitsbuffer, 5, 0x1f);/* Reserved (reserved for future use) */bits_write (& bitsbuffer, 3, 0);/* stuffing length */return 0 ;}

/***** @ Remark: encapsulation of the Sys header. The specific data in it has been filled in with a placeholder. Refer to the standard * @ Param: pdata [in] fill in the PS header data address * @ return: 0 success, others failed */INT gb28181_make_sys_header (char * pdata) {bits_buffer_s bitsbuffer; bitsbuffer. I _size = sys_hdr_len; bitsbuffer. I _data = 0; bitsbuffer. I _mask = 0x80; bitsbuffer. p_data = (unsigned char *) (pdata); memset (bitsbuffer. p_data, 0, sys_hdr_len);/* system header */bits_write (& bitsbuffer, 32, 0x000001bb);/* Start Code */bits_write (& bitsbuffer, 16, SYS_HDR_LEN-6 ); /* header_length indicates the length after the next byte, and the relevant header is also the meaning */bits_write (& bitsbuffer, 1, 1);/* marker_bit */bits_write (& bitsbuffer, 22,500 00);/* rate_bound */bits_write (& bitsbuffer, 1, 1);/* marker_bit */bits_write (& bitsbuffer, 6, 1 ); /* audio_bound */bits_write (& bitsbuffer, 1, 0);/* fixed_flag */bits_write (& bitsbuffer, 1, 1);/* csps_flag */bits_write (& bitsbuffer, 1, 1);/* system_audio_lock_flag */bits_write (& bitsbuffer, 1, 1);/* system_video_lock_flag */bits_write (& bitsbuffer, 1, 1 ); /* marker_bit */bits_write (& bitsbuffer, 5, 1);/* video_bound */bits_write (& bitsbuffer, 1, 0 ); /* DIF from mpeg1 */bits_write (& bitsbuffer, 7, 0x7f);/* reserver * // * audio stream bound */bits_write (& bitsbuffer, 8, 0xc0 ); /* stream_id */bits_write (& bitsbuffer, 2, 3);/* marker_bit */bits_write (& bitsbuffer, 1, 0);/* pstd_buffer_bound_scale */bits_write (& bitsbuffer, 13,512);/* pstd_buffer_size_bound * // * Video Stream bound */bits_write (& bitsbuffer, 8, 0xe0);/* stream_id */bits_write (& bitsbuffer, 2, 3 ); /* marker_bit */bits_write (& bitsbuffer, 1, 1);/* pstd_buffer_bound_scale */bits_write (& bitsbuffer, 13,204 8);/* pstd_buffer_size_bound */return 0 ;}

/***** @ Remark: the encapsulation of the PSM header. The specific data in the header has been filled in. For details, refer to the standard * @ Param: pdata [in] fill in the PS header data address * @ return: 0 success, others failed */INT gb28181_make_psm_header (char * pdata) {bits_buffer_s bitsbuffer; bitsbuffer. I _size = psm_hdr_len; bitsbuffer. I _data = 0; bitsbuffer. I _mask = 0x80; bitsbuffer. p_data = (unsigned char *) (pdata); memset (bitsbuffer. p_data, 0, ps_sys_map_size); bits_write (& bitsbuffer, 24, 0x000001);/* Start Code */B Its_write (& bitsbuffer, 8, 0xbc);/* map Stream ID */bits_write (& bitsbuffer, 16, 18);/* Program Stream Map length */bits_write (& bitsbuffer, 1, 1);/* Current next indicator */bits_write (& bitsbuffer, 2, 3);/* Reserved */bits_write (& bitsbuffer, 5, 0 ); /* Program Stream Map version */bits_write (& bitsbuffer, 7, 0x7f);/* Reserved */bits_write (& bitsbuffer, 1, 1 ); /* marker bit */bits_write (& bitsbuffer, 16, 0);/* progr Ame stream info length */bits_write (& bitsbuffer, 16, 8);/* elementary Stream Map lengthis * // * audio */bits_write (& bitsbuffer, 8, 0x90);/* stream_type */bits_write (& bitsbuffer, 8, 0xc0);/* elementary_stream_id */bits_write (& bitsbuffer, 16, 0 ); /* elementary_stream_info_length is * // * video */bits_write (& bitsbuffer, 8, 0x1b);/* stream_type */bits_write (& bitsbuffer, 8, 0xe0 ); /* elementary_stream_id */bits_wri Te (& bitsbuffer, 16, 0);/* elementary_stream_info_length * // * CRC (2E B9 0f 3d) */bits_write (& bitsbuffer, 8, 0x45 ); /* CRC (24 ~ 31) bits */bits_write (& bitsbuffer, 8, 0xbd);/* CRC (16 ~ 23) bits */bits_write (& bitsbuffer, 8, 0xdc);/* CRC (8 ~ 15) bits */bits_write (& bitsbuffer, 8, 0xf4);/* CRC (0 ~ 7) bits */return 0 ;}

/***** @ Remark: PES header encapsulation. Fill in the specific data in it with a placeholder. Refer to the standard * @ Param: pdata [in] fill in the PS header data address * stream_id [in] code stream type * paylaod_len [in] Load Length * PTS [in] timestamp * DTS [in] * @ return: 0 success, others failed */INT gb28181_make_pes_header (char * pdata, int stream_id, int payload_len, unsigned long pts, unsigned long DTS) {bits_buffer_s bitsbuffer; bitsbuffer. I _size = pes_hdr_len; bitsbuffer. I _data = 0; bitsbuffer. I _mask = 0x80; bitsbuffer. p_data = (unsigned char *) (pdata); memset (bitsbuffer. p_data, 0, pes_hdr_len);/* system header */bits_write (& bitsbuffer, 24, 0x000001);/* Start Code */bits_write (& bitsbuffer, 8, (stream_id);/* streamid */bits_write (& bitsbuffer, 16, (payload_len) + 13 ); /* packet_len * // specify the Data Length in the PES group and the length after this byte and bits_write (& bitsbuffer, 2, 2 ); /* '10' */bits_write (& bitsbuffer, 2, 0);/* scrambling_control */bits_w Rite (& bitsbuffer, 1, 0);/* priority */bits_write (& bitsbuffer, 1, 0);/* data_alignment_indicator */bits_write (& bitsbuffer, 1, 0 ); /* copyright */bits_write (& bitsbuffer, 1, 0);/* original_or_copy */bits_write (& bitsbuffer, 1, 1);/* pts_flag */bits_write (& bitsbuffer, 1, 1);/* dts_flag */bits_write (& bitsbuffer, 1, 0);/* escr_flag */bits_write (& bitsbuffer, 1, 0 ); /* es_rate_flag */bits_write (& bitsbuffer, 1, 0);/* dsm_trick_mode_flag */bits_write (& bitsbuffer, 1, 0);/* additional_copy_info_flag */bits_write (& bitsbuffer, 1, 0 ); /* pes_crc_flag */bits_write (& bitsbuffer, 1, 0);/* pes_extension_flag */bits_write (& bitsbuffer, 8, 10 ); /* header_data_length * // specifies the optional fields contained in the PES group title and the total number of bytes occupied by any padding. The Byte before this field indicates whether there are optional fields. /* Pts, DTS */bits_write (& bitsbuffer, 4, 3);/* '000000' */bits_write (& bitsbuffer, 3, (PTS)> 30) & 0x07);/* PTS [32 .. 30] */bits_write (& bitsbuffer, 1, 1); bits_write (& bitsbuffer, 15, (PTS)> 15) & 0x7fff);/* PTS [29 .. 15] */bits_write (& bitsbuffer, 1, 1); bits_write (& bitsbuffer, 15, (PTS) & 0x7fff);/* PTS [14 .. 0] */bits_write (& bitsbuffer, 1, 1); bits_write (& bitsbuffer, 4, 1);/* '000000' */bits_write (& bitsbuffer, 3, (DTS)> 30) & 0x07);/* DTS [32 .. 30] */bits_write (& bitsbuffer, 1, 1); bits_write (& bitsbuffer, 15, (DTS)> 15) & 0x7fff);/* DTS [29 .. 15] */bits_write (& bitsbuffer, 1, 1); bits_write (& bitsbuffer, 15, (DTS) & 0x7fff);/* DTS [14 .. 0] */bits_write (& bitsbuffer, 1, 1); Return 0 ;}

/***** @ Remark: pack the RTP Header and send data cyclically * @ Param: pdata [in] the data address sent * ndatalen [in] the length of the data sent * mark_flag [in] Mark mark bit * curpts [in] timestamp * Basic of the ppacker [in] data packet information * @ return: 0 success, others failed */INT evaluate (char * databuff, int ndatalen, int mark_flag, data_info_s * ppacker) {int nres = 0; int nplayloadlen = 0; int nsendsize = 0; char szrtphdr [rtp_hdr_len]; memset (szrtphdr, 0, rtp_hdr_len); If (ndata Len + rtp_hdr_len <= rtp_max_packet_buff) // 1460 the ppacker pointer originally has a buffer data cache of 1460 size {// after a frame of data is sent, mark the position 1gb28181_make_rtp_header (szrtphdr, (mark_flag = 1 )? 1: 0), ++ ppacker-> u16cseq, (ppacker-> s64curpts/300), ppacker-> u32ssrc); memcpy (ppacker-> szbuff, szrtphdr, rtp_hdr_len ); memcpy (ppacker-> szbuff + rtp_hdr_len, databuff, ndatalen); nret = senddatabuff (databuff, rtp_hdr_len + nsendsize, ppacker); If (nres! = (Rtp_hdr_len + ndatalen) {printf ("UDP send error! \ N "); Return-1 ;}} else {nplayloadlen = rtp_max_packet_buff-rtp_hdr_len; // only the data length that can be sent each time except the RTP Header gb28181_make_rtp_header, ++ ppacker-> u16cseq, (ppacker-> s64curpts/100), ppacker-> u32ssrc), memcpy (ppacker-> szbuff + rtp_hdr_len, databuff, nplayloadlen ); nret = senddatabuff (databuff, rtp_hdr_len + nsendsize, ppacker); If (nres! = (Rtp_hdr_len + nplayloadlen) {printf ("UDP send error! \ N "); Return-1;} ndatalen-= nplayloadlen; // databuff + = (nplayloadlen-rtp_hdr_len); databuff + = nplayloadlen; // indicates that the previous data has been sent out databuff-= rtp_hdr_len; // It is used to store the RTP Header while (ndatalen> 0) {If (ndatalen <= nplayloadlen) {// after a frame of data is sent, set the mark to gb28181_make_rtp_header (databuff, mark_flag, ++ ppacker-> u16cseq, (ppacker-> s64curpts/100 ), ppacker-> u32ssrc); nsendsize = ndatalen;} else {gb28181_make_rtp_header (databuff, 0, ++ ppacker-> u16cseq, (ppacker-> s64curpts/100), ppacker-> u32ssrc); nsendsize = nplayloadlen;} nret = senddatabuff (databuff, latency + nsendsize, ppacker); If (nres! = (Rtp_hdr_len + nsendsize) {printf ("UDP send error! \ N "); Return-1;} ndatalen-= nsendsize; databuff + = nsendsize; // Since the buffer pointer has been moved backward once after the RTP Header length, // when the RTP packet is sent cyclically, you only need to move the raw data forward to the length, this is where the buffer pointer actually points to the position. // databuff refers to the raw data with the same RTP length to the position.} return 0 ;}

Another very important macro definition is to define it as a macro because it is frequently called. Its function is to push the 8-bit bitwise values of one byte into data one by one in a loop, to prevent errors in collation caused by exaggerated bytes, the specific implementation is as follows. In fact, VLC source code is used for implementation, which is also a benefit of reading the source code, it makes good use of the advanced and convenient functional code modules.

# Define ps_hdr_len 14 # define sys_hdr_len 18 # define psm_hdr_len 24 # define pes_hdr_len 19 # define rtp_hdr_len 12/***** @ remark: the incoming data is pushed into the data one by position * @ Param: buffer [in] the buffer * count [in] of the pushed data the number of digits * bits [in] of the pushed data */# define bits_write (buffer, Count, BITs) {bits_buffer_s * p_buffer = (buffer); int I _count = (count); uint64_t I _bits = (BITs); While (I _count> 0) {I _count --; if (I _bits> I _count) & 0 X01) {p_buffer-> p_data [p_buffer-> I _data] | = p_buffer-> I _mask;} else {p_buffer-> p_data [p_buffer-> I _data] & = ~ P_buffer-> I _mask;} p_buffer-> I _mask> = 1;/* after the first byte is completed, the second byte is operated */If (p_buffer-> I _mask = 0) /* after completing the 8-bit loop of a byte, start the next bit again */{p_buffer-> I _data ++; p_buffer-> I _mask = 0x80 ;}}}

I forgot to post the RTP Encapsulation Header. In this addition, if you do not need RTP, you can directly block the gb28181_make_rtp_header function interface in the gb28181_send_rtp_pack function interface. Of course, pay attention to some issues, the corresponding buffer pointer does not need to move the RTP Header length!

int gb28181_make_rtp_header(char *pData, int marker_flag, unsigned short cseq, long long curpts, unsigned int ssrc){bits_buffer_s  bitsBuffer;if (pData == NULL)return -1;bitsBuffer.i_size = RTP_HDR_LEN;bitsBuffer.i_data = 0;bitsBuffer.i_mask = 0x80;bitsBuffer.p_data =(unsigned char *)(pData);memset(bitsBuffer.p_data, 0, RTP_HDR_SIZE);bits_write(&bitsBuffer, 2, RTP_VERSION);/* rtp version */bits_write(&bitsBuffer, 1, 0);/* rtp padding */bits_write(&bitsBuffer, 1, 0);/* rtp extension */bits_write(&bitsBuffer, 4, 0);/* rtp CSRC count */bits_write(&bitsBuffer, 1, (marker_flag));/* rtp marker  */bits_write(&bitsBuffer, 7, 96);/* rtp payload type*/bits_write(&bitsBuffer, 16, (cseq));/* rtp sequence  */bits_write(&bitsBuffer, 32, (curpts)); /* rtp timestamp  */bits_write(&bitsBuffer, 32, (ssrc)); /* rtp SSRC  */return 0;}

4. encapsulation related experiences

The blogger has verified step-by-step that RTP, PS, or RTP + PS encapsulation can be properly previewed. In fact, there is really no theoretical knowledge about this encapsulation. If we look at all the standards, we only need to carefully look at the standard step by step and analyze the fields of each encapsulation, there is no problem, of course, there are also many small problems that have plagued me for a long time, but all of them are due to lack of standard attention or misunderstanding. Now, I just simply posted my own code and shared it. I hope that the development of the Code will take less detours. If you have any questions or have a better way, you can communicate with each other.

Code implementation for PS encapsulation of h264 code streams