FFmpeg Introduction Summary (i)--YUV format example

1.ffmpeg Introduction

FFmpeg is an open source free cross-platform video and audio streaming solution that belongs to free software with LGPL/GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library Libavcodec, in order to ensure high portability and codec quality, Libavcodec Many codec are developed from scratch. FFmpeg is an open-source computer program that can be used to record, convert digital audio, video, and turn it into a stream. It includes the current leading audio/video coding library Libavcodec. FFmpeg is developed under Linux, but it can be compiled in most operating systems, including Windows.

----------------------

Some basic concepts of ffmpeg

----------------------

Container: In the audio and video "container", generally refers to a specific format of the multimedia file, which indicates the included audio and video, subtitles and other related information;
Stream: Can be understood as exploded, pure audio data stream or video data stream or subtitle stream, etc.;
Frame: Refers to an uncompressed complete data unit in the stream;
Packet: Is the structure of storing compressed data, the video stream is usually a frame of compression, and in the audio stream may be compressed multiple frame;
Codec: Codec;

Sample rate: The number of sampling points recorded per second;

----------------------

Composition of the FFMPEG structure

----------------------

The ffmpeg consists mainly of the following libraries:

· Libavcodec:encoding/decoding Library (codec libraries)

· libavfilter:graph-based Frame Editing library (filter libraries, similar to the "" feature of the DShow, but require the input/output interface of the filter to be matched)

· LIBAVFORMAT:I/O and Muxing/demuxing library (IO and decomposition/merge streams)

· Libavdevice:special devices muxing/demuxing Library (e.g. open camera, screen recording, etc.)

· Libavutil:common Utility Library (small function libraries, memory allocation, release)

· Libswresample:audio resampling, format conversion and mixing

· Libpostproc:post Processing library (post-processing. Not yet clear)

· Libswscale:color Conversion and Scaling library (transcoding and scale transformation)

The FFmpeg package will generate three executables (which can be run from the command line) after it has been compiled:

· Ffmpeg:ffmpeg is used to process media files;

· Ffserver:ffserver is an HTTP streaming media server;

· Ffplay:ffplay is a simple SDL-based player. 2. Video YUV Format Introduction

YUV format, generally used y,u,v ratio to represent different formats, such as YUV444 means that the three are more than the 4:4:4, that is, a point data points, y,u,v space is the same size. At present, there are mainly the following ratios, note that all formats have a Y ratio of 4, which takes up one byte, indicating no reduction in sampling. In different formats, only the sampled values of the Uvs are reduced.

4:4:4: Indicates that the chroma value (UV) does not reduce sampling. That is, the Y,U,V account for one byte, totaling 3 bytes.

4:2:2: Represents half of the UV component sampling, such as the first pixel sampling y,u, the second pixel sampling y,v, and so on, so that each point occupies 2 bytes. Two pixels to form a macro pixel.

4:2:0: This sample does not mean that there is only Y,CB and no CR component, where 0 of the U,v component is sampled once. For example, the first line sampled 4:2:0, the second line sampled 4:0:2, and so on ... In this sampling mode, an average of 1.5 bytes per pixel is consumed.

4:2:2 Example

If the original data three pixels is [Y0 U0 V0], [Y1 U1 v1],[y2 U2 v2],[y3 U3 V3]

After 4:2:2 sampling, the data becomes Y0 U0, Y1 V1, Y2 u2,y3 V3

[Y0 U0 v1],[y1 U0 v1],[y2 U2 V3], [Y3 U3 V2] When restored due to some data loss

4:2:0 Example

The following eight pixels are: [Y0 U0 V0] [Y1 U1 V1] [Y2 U2 V2] [Y3 U3 V3] [Y5 U5 V5] [Y6 U6 V6] [y7u7 V7] [Y8 U8 V8]

The stored streams are: Y0 U0, Y1, Y2 U2, Y3,y5 V5, Y6,y7 V7, Y8

Mapped pixel points are: [Y0 U0 V5] [Y1 U0 V5] [Y2 U2 V7] [Y3 U2 V7] [Y5 U0 V5] [Y6 U0 V5] [Y7 U2 V7]

In addition to 4:4:4 sampling, the rest of the sample after the signal re-restore display, will lose some of the UV data, can only be used to complement the data, but the human eye is not sensitive to UV, so the overall sense of loss is small.

Figure 1 YUV420SP storage format

Figure 2 yuv420p storage format

Ps:ffmpeg, the original image data includes two kinds: planar and packed. Planar is to separate several components, such as YUV420, Data[0] dedicated to save Y,data[1] dedicated to save v. While packed is packaged, all data is present in data[0]. 3.YUV Format example "Rotate 90°yuv420 image Clockwise":

  public static void rotateYUV420 (byte[] src,byte[] des,int width,int height, String type) {//original is stored in row order
		
		int WH = width * height;
		int k = 0; Rotate y component (stored in column order by original) for (int i=0;i<width;i++) {for (int j=0;j

Ps:ffmpeg Library interfaces are C functions, in the CPP file called FFmpeg function to note that because the C + + language supports function overloading, C language does not support function overloading, the function is compiled by the C + + compiler in the library name and C language is different (mangled  name), a library written in C, if you want to be able to use it at the same time, the header file should be prefixed with the following code

#ifdef  __cplusplus
extern  "C"  {
# endif 

...
#ifdef  __cplusplus
}
#endif