H265關於RTP封包__rtp

首先來介紹下h265（HEVC）nal單元頭，與h264的nal層相比，h265的nal unit header有兩個位元組構成，如下圖所示：

從圖中可以看出hHEVC的nal包結構與h264有明顯的不同，hevc加入了nal所在的時間層的ＩＤ，取去除了nal_ref_idc，此資訊合并到了naltype中，通常情況下F為0，layerid為0,TID為1。

    nal單元的類型有如下幾種：

[cpp] view plain copy 在CODE上查看代碼片派生到My Code片
enum NalUnitType
{
NAL_UNIT_CODED_SLICE_TRAIL_N = 0, // 0
NAL_UNIT_CODED_SLICE_TRAIL_R, // 1

NAL_UNIT_CODED_SLICE_TSA_N, // 2
NAL_UNIT_CODED_SLICE_TLA, // 3 // Current name in the spec: TSA_R

NAL_UNIT_CODED_SLICE_STSA_N, // 4
NAL_UNIT_CODED_SLICE_STSA_R, // 5

NAL_UNIT_CODED_SLICE_RADL_N, // 6
NAL_UNIT_CODED_SLICE_DLP, // 7 // Current name in the spec: RADL_R

NAL_UNIT_CODED_SLICE_RASL_N, // 8
NAL_UNIT_CODED_SLICE_TFD, // 9 // Current name in the spec: RASL_R

NAL_UNIT_RESERVED_10,
NAL_UNIT_RESERVED_11,
NAL_UNIT_RESERVED_12,
NAL_UNIT_RESERVED_13,
NAL_UNIT_RESERVED_14,
NAL_UNIT_RESERVED_15, NAL_UNIT_CODED_SLICE_BLA, // 16 // Current name in the spec: BLA_W_LP
NAL_UNIT_CODED_SLICE_BLA, // 16 // Current name in the spec: BLA_W_LP
NAL_UNIT_CODED_SLICE_BLANT, // 17 // Current name in the spec: BLA_W_DLP
NAL_UNIT_CODED_SLICE_BLA_N_LP, // 18
NAL_UNIT_CODED_SLICE_IDR, // 19 // Current name in the spec: IDR_W_DLP
NAL_UNIT_CODED_SLICE_IDR_N_LP, // 20
NAL_UNIT_CODED_SLICE_CRA, // 21
NAL_UNIT_RESERVED_22,
NAL_UNIT_RESERVED_23,

NAL_UNIT_RESERVED_24,
NAL_UNIT_RESERVED_25,
NAL_UNIT_RESERVED_26,
NAL_UNIT_RESERVED_27,
NAL_UNIT_RESERVED_28,
NAL_UNIT_RESERVED_29,
NAL_UNIT_RESERVED_30,
NAL_UNIT_RESERVED_31,

NAL_UNIT_VPS, // 32
NAL_UNIT_SPS, // 33
NAL_UNIT_PPS, // 34
NAL_UNIT_ACCESS_UNIT_DELIMITER, // 35
NAL_UNIT_EOS, // 36
NAL_UNIT_EOB, // 37
NAL_UNIT_FILLER_DATA, // 38
NAL_UNIT_SEI, // 39 Prefix SEI
NAL_UNIT_SEI_SUFFIX, // 40 Suffix SEI
NAL_UNIT_RESERVED_41,
NAL_UNIT_RESERVED_42,
NAL_UNIT_RESERVED_43,
NAL_UNIT_RESERVED_44,
NAL_UNIT_RESERVED_45,
NAL_UNIT_RESERVED_46,
NAL_UNIT_RESERVED_47,
NAL_UNIT_UNSPECIFIED_48,
NAL_UNIT_UNSPECIFIED_49,
NAL_UNIT_UNSPECIFIED_50,
NAL_UNIT_UNSPECIFIED_51,
NAL_UNIT_UNSPECIFIED_52,
NAL_UNIT_UNSPECIFIED_53,
NAL_UNIT_UNSPECIFIED_54,
NAL_UNIT_UNSPECIFIED_55,
NAL_UNIT_UNSPECIFIED_56,
NAL_UNIT_UNSPECIFIED_57,
NAL_UNIT_UNSPECIFIED_58,
NAL_UNIT_UNSPECIFIED_59,
NAL_UNIT_UNSPECIFIED_60,
NAL_UNIT_UNSPECIFIED_61,
NAL_UNIT_UNSPECIFIED_62,
NAL_UNIT_UNSPECIFIED_63,
NAL_UNIT_INVALID,
};
下面接收下fu分組打包方式，fu分組包頭格式如下：

fus包頭包含了兩個位元組的payloadhdr，一個位元組的fu header，fu header與h264一樣，結構如下圖，包含開始位（1b）、停止位(1b)、futype(6b)

paylodhdr兩個自己的賦值，其實就是把hevc幀資料的nal unit header的naltype替換為49即可，下面是從ffmpeg源碼中截取出來的fu打包方式程式碼片段：

[cpp] view plain copy 在CODE上查看代碼片派生到My Code片
static void nal_send(AVFormatContext *ctx, const uint8_t *buf, int len, int last_packet_of_frame)
{
RTPMuxContext *rtp_ctx = ctx->priv_data;
int rtp_payload_size = rtp_ctx->max_payload_size - RTP_HEVC_HEADERS_SIZE;
int nal_type = (buf[0] >> 1) & 0x3F;

    /* send it as one single NAL unit? */      if (len <= rtp_ctx->max_payload_size) //小於對定的最大值時，直接發送（最大值一般小於mtu）   {       /* use the original NAL unit buffer and transmit it as RTP payload */      ff_rtp_send_data(ctx, buf, len, last_packet_of_frame);  }  else //大於最大值時進行fu分組發送  {      /*                 create the HEVC payload header and transmit the buffer as fragmentation units (FU)                 0                   1                 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                 |F|   Type    |  LayerId  | TID |                 +-------------+-----------------+                     F       = 0                     Type    = 49 (fragmentation unit (FU))                     LayerId = 0                     TID     = 1     */      rtp_ctx->buf[0] = 49 << 1;      rtp_ctx->buf[1] = 1;         //此處為paylaodhdr,規範賦值應該是替換hevc資料nal 的payloadhdr的type          //rtp_ctx->buf[0] = (buf[0] &0x81) | (49<<1);     //rtp_ctx->buf[1] = buf[1]      /*         create the FU header         0 1 2 3 4 5 6 7         +-+-+-+-+-+-+-+-+         |S|E|  FuType   |         +---------------+             S       = variable             E       = variable             FuType  = NAL unit type     */      rtp_ctx->buf[2] = nal_type;      /* set the S bit: mark as start fragment */      rtp_ctx->buf[2] |= 1 << 7;           /* pass the original NAL header */      //此處要注意，當是分組的第一報資料時，應該覆蓋掉前兩個位元組的資料，h264要覆蓋前一個位元組的資料，即是第一包要去除hevc幀資料的paylaodhdr      buf += 2;      len -= 2;         while (len > rtp_payload_size)       {          /* complete and send current RTP packet */              memcpy(&rtp_ctx->buf[RTP_HEVC_HEADERS_SIZE], buf, rtp_payload_size);          ff_rtp_send_data(ctx, rtp_ctx->buf, rtp_ctx->max_payload_size, 0);              buf += rtp_payload_size;          len -= rtp_payload_size;              /* reset the S bit */              rtp_ctx->buf[2] &= ~(1 << 7);      }       /* set the E bit: mark as last fragment */       rtp_ctx->buf[2] |= 1 << 6;       /* complete and send last RTP packet */       memcpy(&rtp_ctx->buf[RTP_HEVC_HEADERS_SIZE], buf, len);      ff_rtp_send_data(ctx, rtp_ctx->buf, len + 2, last_packet_of_frame);  }  

}

通過rtp發送hevc視頻資料，當hevc幀資料大於mtu時，應該進行fu分組發送，從上面代碼流程就是對超過max_payload_size資料進行fu分組的流程，這個h264 fu-A很類似，很容易理解。

參考規範：

https://tools.ietf.org/html/draft-ietf-payload-rtp-h265-14

ffmpeg相關代碼

https://www.ffmpeg.org/doxygen/2.5/rtpenc__hevc_8c_source.html