Conversion of FLV stream to standard h264 and AAC in rtmp

During this period of time, the work needs are converted from FLV stream to standard h264 and aac on rtmp. The server is an open-source project crtmpserver, client flex writing, video encoding h264, and audio encoding AAC, some protocol-related items are recorded as follows.

 

1. Analyze FLV data

Let's take a FLV file to analyze the FLV data format.

FLV standard document http://www.cnblogs.com/chef/archive/2012/07/18/2597003.html

FLV file analyzer flvprasehttp: // www.cnblogs.com/chef/archive/2012/07/18/2596930.html

The focus of this article is not here. I believe that you can understand the FLV data format against the FLV standard documentation, and I strongly recommend that you first finish this step.

Ii. Analyze the upstream h264 video stream of rtmp

On the server, the uplink video is saved as a binary file (note that data must be saved in binary format). For example, the binary data displayed in the image contains 16 columns in each row, that is, from 0 to F, if the display is incomplete, open the image separately)

I used notepad ++ (and installed the binary viewing plug-in)

If you do the first step, it is not difficult to see that the FLV video stream in rtmp is a next-to-next video tag-that is, the header information is removed from the FLV tag, and only the video TAG content is retained.

We will analyze them one by one based on the FLV standard documents.

17:1-keyframe 7-avc

00: AVC sequence header -- AVC packet type

00 00 00: composition time, AVC, all 0, meaningless

Because AVC packet type = AVC sequence header, the following is the content of avcdecoderconfigurationrecord.

Configurationversion = 01

Avcprofileindication = 42

Profile_compatibility = 00

Avclevelindication = 1f

Lengthsizeminusone = ff -- the number of bytes used for the long data of the NALU package in FLV (lengthsizeminusone & 3) + 1. In actual test, FF is found, and 4 is calculated, this data will be mentioned below

Numofsequenceparametersets = e1 -- number of SPS, numofsequenceparametersets & 0x1f. during actual testing, the total value is E1 and the calculation result is 1.

Sequenceparametersetlength = 00 31 -- SPS length, 2 bytes, Calculation Result 49

Sequenceparametersetnalunits = 67 42 80 1f 96 54 05 01 ed 80 A8 40 00 00 00 00 00 40 00 07 B8 00 00 20 00 00 03 01 01 01 FC 63 8C 00 00 10 00 00 03 00 80 00 Fe 31 C3 B4 24 4D 40 -- SPS, for the 49 bytes just calculated, SPS contain video length and width information.

Numofpictureparametersets = 01 -- number of PPS. In actual test, the total value is e1, and the calculation result is 1.

Pictureparametersetlength = 00 04 -- PPS Length

Pictureparametersetnalunits = 68 ce 35 20 -- PPS

Next we have a new set of videotag data.

17:1-keyframe 7-avc

01: AVC nalu

00 00 00: composition time, AVC, all 0, meaningless

Because avcpacket type = AVC nalu, one or more nalu

Each NALU package has a (lengthsizeminusone & 3) + 1-byte nal package length description (as mentioned above, remember). The preceding calculation result is 4 bytes.

00 00 00 02: 2 -- NALU Length

09 10: nal package

Insert a little knowledge about NALU. The first five digits of each NALU indicate the type of the nal package, that isNAL nal_unit_type

# Define nalu_type_slice 1
# Define nalu_type_dpa 2
# Define nalu_type_dpb 3
# Define nalu_type_dpc 4
# Define nalu_type_idr 5
# Define nalu_type_sei 6
# Define nalu_type_sps 7
# Define nalu_type_pps 8
# Define nalu_type_aud 9 // access Separator
# Define nalu_type_eoseq 10
# Define nalu_type_eostream 11
# Define nalu_type_fill 12
 

09 & 0x1f = 9, access unit Separator

In the preceding parsing, the SPS header byte is 67,67 & 0x1f = 7, and the PPS header byte is 68, 68 & 0x1f = 8.

00 00 00 29: The subsequent nal package length is 41

06 00 11 80 00 af C8 00 00 03 00 00 00 00 af C8 00 00 03 00 40 01 0C 00 00 00 00 03 00 00 90 80 08 00 00 03 00 08 80: 06 & 0x1f = 6 -- sei

00 00 3C D0: Subsequent nal package Length

65 88 80 ...... : 65 & 0x1f = 5 -- I frame data

The analysis of this video tag is over. Next we will discuss the p-frame data corresponding to this I-frame. See

 

Look at the line 3d80, and the previous content until 53 4f 7f is the content of the previous video tag, that is, the data pulling of the I frame mentioned above 65 88 80, 27 is a new video tag.

27:2-inter frame is P frame, 7-codecid = AVC

01: avcpacket type = AVC nalu

00 00 00: composition time, AVC, all 0, meaningless

00 00 00 02 09 30: Pull the two-byte nal package and access unit separator as analyzed above

00 00 byte nal package

06 01 0C 00 00 80 00 00 90 80 18 00 00 03 00 08 80: 06 & 0x1f = 6-sei

00 00 46 85: Data Length of the nal package

41 9A 02 ...... : 41 & 0x1f = 1, P frame data

 

Iii. Conversion

In summary, the FLV h264 stream is sequential

1. A video tag contains the following information: SPS, PPS, access unit separator, SEI, And I frame package.

2. One or more video tags, including the access unit separator, SEI, and P frame packages.

Loop 1, 2

Here we need to explain that, in this step of conversion, we only need to getAll, one by oneThe Nal package is fine. As for I frame, P frame and other types, we don't need to worry about it. Here we just want to better analyze the data.

The NALU and NALU of h264 are separated by 00 00 01 (or 00 00 01). The format after h264 is

1. 00 00 00 01 SPS 00 00 00 01 PPS 00 00 01 access unit separator 00 00 00 01 01 sei 00 00 00 01 I frame 00 00 00 01 P frame 00 00 00 01 P frame ...... (The number of P frames is not fixed)

Loop 1

The access unit separator and SEI are not mandatory. When h264 is written to a file in binary format, it can be played using elecard streameye,

Http://www.cnblogs.com/chef/archive/2012/07/18/2597008.html

Note: I have been in touch with the audio and video industry for a short time, so the content in this article is not necessarily correct. If you find any problems, please point out thank you. In addition, there are a few references to some materials on the network, but the original source of the data cannot be found, and it is simply not written from where.