FLV format details

Overview

Flash video(AbbreviationFLVIs a popular network format. Most video sharing websites at Home and Abroad use this format.

File structure

From the perspective of the entire file, FLV is composedThe FLV HeaderAndThe FLV file bodyComposition.

1. The FLV Header

Field	Type	Comment
Signature	Ui8	Signature byte always 'F' (0x46)
Signature	Ui8	Signature byte always 'l' (0x4c)
Signature	Ui8	Signature byte always 'V' (0x56)
Version	Ui8	File version (for example, 0x01 for FLV version 1)
Typeflagsreserved	UB [5]	Shall be 0
Typeflagsaudio	UB [1]	1 = audio tags are present
Typeflagsreserved	UB [1]	Shall be 0
Typeflagsvideo	UB [1]	1 = video tags are present
Dataoffset	Ui32	The length of this header in bytes

 

Signature: The first three bytes of the FLV file are fixed'F''L''V', Used to identify the file in FLV format. During format detection,

If the first three bytes are found to be "FLV", it is considered as a FLV file.

Version: 4th bytes indicates the FLV version number.

Flags: 5th bits and 0th bits in 2nd bytes indicate the existence of video and audio respectively. (1 indicates existence, 0 indicates nonexistent)

Dataoffset: The last four bytes indicate the length of the FLV header.

2. The FLV file body

Field	Type	Comment
Previoustagsize0	Ui32	Always 0
Tag1	Flvtag	First tag
Previoustagsize1	Ui32	Size of previous tag, including its header, in bytes. For FLV Version1, This value is 11 plus the datasize of the previous tag.
Tag2	Flvtag	Second tag
...	...	...
PreviousTagSizeN-1	Ui32	Size of second-to-last tag, including its header, in bytes.
Tagn	Flvtag	Last tag
Previoustagsizen	Ui32	Size of last tag, including its header, in bytes

 

FLV HeaderThenFLV file body.

FLV file bodyIs composed of a series of back-Pointers + tags. Back-pointers is 4 bytes of data, indicating the size of the previous tag.

 

FLV tag Definition

The data in the FLV file is composed of tags. The data in the tag may be video, audio, and scripts.

The following table lists the tag structure:

1. flvtag

Field	Type	Comment
Reserved	UB [2]	Reserved for FMS, shocould be 0
Filter	UB [1]	Indicates if packets are filtered. 0 = No pre-processing required. 1 = pre-processing (such as decryption) of the packet is Required before it can be rendered. Shall be 0 in unencrypted files, and 1 for encrypted tags. See Annex F. FLV encryption for the use of filters.
Tagtype	UB [5]	Type of contents in this tag. The following types are Defined: 8 = audio 9 = video 18 = script data
Datasize	Ui24	Length of the message. number of bytes after streamid End of tag (equal to length of the tag-11)
Timestamp	Ui24	Time in milliseconds at which the data in this tag applies. This value is relative to the first tag in the FLV file, which Always has a timestamp of 0.
Timestampextended	Ui8	Extension of the timestamp field to form a si32 value. This Field represents the upper 8 bits, while the previous Timestamp Field represents the lower 24 bits of the time in Milliseconds.
Streamid	Ui24	Always 0.
Audiotagheader	If tagtype = 8 Audiotagheader
Videotagheader	If tagtype = 9 Videotagheader
Encryptionheader	If filter = 1 Encryptiontagheader
Filterparams	If filter = 1 Filterparams
Data	If tagtype = 8 Audiodata If tagtype = 9 Videodata If tagtype = 18 Scriptdata	Data specific for each media type.

 

Tagtype: The first five digits of the 1st bytes in the tag indicate the data type in the tag, 8 = audio, 9 = video, 18 = script data.

Datasize: The length of data after streamid.

TimestampAndTimestampextendedThe PTS information of the tag package data. Remember that the value of timestampextended was not taken into account when fvl Demux was started, and the default value of timestamp is PTS, later, the phenomenon of frame skipping occurs on the screen. Later, I carefully read the document and found that the real data PTS is PTS = timestamp | timestampextended <24.

StreamidThe subsequent data is different in each format. The subsequent format is described in detail.

Audio tags

IfTagtype= 8 indicates that the tag is audio.

StreamidThe subsequent data indicates yes.Audiotagheader,AudiotagheaderThe structure is as follows:

Field	Type	Comment
Soundformat	UB [4]	Format of sounddata. the following values are defined: 0 = Linear PCM, platform endian 1 = ADPCM 2 = MP3 3 = Linear PCM, little endian 4 = nellymoser 16 kHz mono 5 = nellymoser 8 kHz mono 6 = nellymoser 7 = g.711 A-Law logarithmic PCM 8 = g.711 Mu-Law logarithmic PCM 9 = Reserved 10 = AAC 11 = speex 14 = MP3 8 kHz 15 = device-specific sound Formats 7, 8, 14, and 15 are reserved. AAC is supported in Flash Player 9, 0, 115,0 and higher. Speex is supported in Flash Player 10 and higher.
Soundrate	UB [2]	Sampling rate. The following values are defined: 0 = 5.5 kHz 1 = 11 kHz 2 = 22 kHz 3 = 44 kHz
Soundsize	UB [1]	Size of each audio sample. This parameter only pertains Uncompressed formats. Compressed formats always decode To 16 bits internally. 0 = 8-bit Samples 1 = 16-bit Samples
Soundtype	UB [1]	Mono or stereo sound 0 = mono sound 1 = stereo sound
Aacpackettype	If soundformat = 10 Ui8	The following values are defined: 0 = AAC sequence Header 1 = AAC raw

 

AudiotagheaderIs followedStreamidThe first byte contains the basic information such as the audio type and sampling rate. The columns in the table are very clear.

AudiotagheaderFollowedAudiodataData, that is, audio payload, but here is a special case. If the audio format (soundformat) is 10 = AAC,AudiotagheaderOne more byte of dataAacpackettypeThis field is used to representAacaudiodataType: 0 = AAC sequence header, 1 = AAC raw.

Field	Type	Comment
Data	If aacpackettype = 0 audiospecificconfig	The audiospecificconfig is defined in ISO14496-3. Note that this is not the same as the contents of the ESDs box from an MP4/f4v file.
	Else if aacpackettype = 1 raw AAC frame data in ui8 []	Audio Payload

AAC sequence HeaderThat is, includingAudiospecificconfig,AudiospecificconfigContains more detailed Audio Information. The definition of audiospecificconfig is as follows:ISO14496-3Medium1.6.2.1 audiospecificconfig. In additionFFmpegThere are functions for parsing audiospecificconfig,Ff_mpeg4audio_get_config(). Let's take a look at it for a better understanding.

AAC rawThis includes the audio es stream, that is, audio payload.

In a FLV fileAAC sequence HeaderThis kind of package only appears once, and it is the first audio tag. Why should we mention this tag, because when we were doing flvdemux, if it was AAC audio, you need to add 7 bytes to the front of each AAC es stream.AdstHeader,AdstThe audio format will be interpreted in detail. This is a common decoder format, that is, the pure es stream of AAC must be packaged into an AAC file in the adst format before the decoder can play normally. when adst is packagedSamplingfrequencyindexThe most accurate information about samplingfrequencyindex isAudiospecificconfigSo the audiospecificconfig is parsed and samplingfrequencyindex is obtained.

In this step, you can extract the audio information and data from the FLV file and send it to the audio decoder for normal playback.

Video tags

IfTagtype= 9 indicates that the tag is video.

StreamidThe subsequent data indicates yes.Videotagheader,VideotagheaderThe structure is as follows:

Field	Type	Comment
Frame Type	UB [4]	Type of video frame. The following values are defined: 1 = Key Frame (for AVC, A seekable frame) 2 = inter frame (for AVC, a non-seekable frame) 3 = disposable inter frame (H.263 only) 4 = generated key frame (reserved for server use only) 5 = video info/command Frame
Codecid	UB [4]	Codec identifier. the following values are defined: 2 = Sorenson H.263 3 = screen video 4 = on2 VP6 5 = on2 VP6 with alpha channel 6 = screen video version 2 7 = AVC
Avcpackettype	If codecid = 7 Ui8	The following values are defined: 0 = AVC sequence Header 1 = AVC nalu 2 = AVC end of sequence (lower level NALU sequence ender is not required or supported)
Compositiontime	If codecid = 7 Si24	If avcpackettype = 1 Composition Time Offset Else 0 See ISO 14496-12, 8.15.3 for an explanation of Composition Times. The offset in an FLV file is always in milliseconds.

VideotagheaderIs followedStreamidThe first byte contains the video frame type and the basic information of the video codecid. The column in the table is very clear.

VideotagheaderFollowedVideodataThe data, that is, video payload. Of course, just like audio AAC, there is also a special case here that if the video format is AVC (H.264,Videotagheader4 more bytes of information.

Avcpackettype and compositiontime. Avcpackettype indicates the followingVideodata (avcvideopacket)Content:

If avcpackettype = 0Avcdecoderconfigurationrecord(AVC sequence header)
If avcpackettype = 1One or more nalus (full frames are required)

Avcdecoderconfigurationrecord. contains important SPS and PPS information related to H.264 decoding. Before sending data to the AVC decoder, the SPS and PPS information must be sent. Otherwise, the decoder cannot be decoded normally. In addition, before the decoder stops and starts again, such as seek and fast forward and return status switch, the SPS and PPS information must be re-sent. avcdecoderconfigurationrecord usually appears once in the FLV file, that is, the first video tag.

Avcdecoderconfigurationrecord is defined inISO 14496-15, 5.2.4.1, which is not detailed here,

Scriptdata

IfTagtype= 18 indicates that the tag isScript.

The scriptdata structure is very complex, and many format types are defined. Each type corresponds to a structure.

Field	Type	Comment
Type	Ui8	Type of the scriptdatavalue. The following types are defined: 0 = Number 1 = Boolean 2 = string 3 = Object 4 = movieclip (reserved, not supported) 5 = NULL 6 = undefined 7 = reference 8 = ECMA Array 9 = Object End marker 10 = strict Array 11 = Date 12 = long string
Scriptdatavalue	If type = 0 Double If type = 1 Ui8 If type = 2 Scriptdatastring If type = 3 Scriptdataobject If type = 7 Ui16 If type = 8 Scriptdataecmaarray If type = 10 Scriptdatastrictarray If type = 11 Scriptdatadate If type = 12 Scriptdatalongstring	Script data value. The Boolean value is (scriptdatavalue =0 ).

The type is described in detail in FLV official documentation.

Onmetadata

Onmetadata isScriptdataThe following table lists important information:

Property name	Type	Comment
Audiocodecid	Number	Audio Codec ID used in the file (see e.4.2.1 for available soundformat values)
Audiodatarate	Number	Audio Bit Rate in kilobits per second
Audiodelay	Number	Delay introduced by the Audio Codec in seconds
Audiosamplerate	Number	Frequency at which the audio stream is replayed
Audiosamplesize	Number	Resolution of a single audio sample
Canseektoend	Boolean	Indicating the last video frame is a key frame
Creationdate	String	Creation date and time
Duration	Number	Total duration of the file in seconds
Filesize	Number	Total size of the file in bytes
Framerate	Number	Number of frames per second
Height	Number	Height of the video in pixels
Stereo	Boolean	Indicating stereo audio
Videocodecid	Number	Video Codec ID used in the file (see e.4.3.1 for available codecid values)
Videodatarate	Number	Video Bit Rate in kilobits per second
Width	Number	Width of the video in pixels

Here, the duration, filesize, video width, height, and other information are very useful for us.

Keyframes

At the time of FLV Demux, it was found thatKeyframeS index is described, but in this structure of FLV, each tag does not have a synchronization header like ts. If noKeyframeIf the S index is used, the effect of seek and fast forward will be very poor, because a tag needs to be read sequentially. Later, I foundKeyframesInformation hidden inScriptdata.

KeyframesAlmost oneUnofficialThe standard, that is, the folk standard. it is hard to see the FLV file format on the Internet, but metadata does not contain the video of the keyframes project. two commonly used tools for operating metadata are flvtool2 and flvmdi, both of which use keyframes as a default metadata project. on the flvmdi home page (http://www.buraks.com/flvmdi/) There is a description:

Keyframes: (Object) This object is added only if you specify the/K switch. 'keyframeworks' is known to flvmdi and if/K switch is not specified, 'keyframework' object will be deleted.
'Keyframes 'object has 2 Arrays: 'filepositions 'and 'times '. both arrays have the same number of elements, which is equal to the number of key frames in the FLV. values in times array are in 'seconds '. each correspond to the timestamp of the N' th key frame. values in filepositions array are in 'bytes '. each correspond to the fileposition of the nth key frame video tag (which starts with byte tag type 9 ).

That is to sayKeyframesThe two contents 'positions' and 'times 'indicate the File Location of the key frame and the PTS of the key frame respectively.KeyframesYou can create your own index, and then perform the seek and fast forward and return operations to quickly and effectively jump to the location of the key frame you want to find for processing.