About the common AYUV formats in RGB, YUY2, Yuyv, Yvyu, UYVY, DirectShow RGB/YUV

Tip: RGB and YUV----excerpt from the DirectShow Practice collection Author: Lu Qiming

Computer color display the principle of color display and color TV, is the use of R (Red), G (Green), B (Blue) additive blending principle: By emitting three different intensity of the electron beam, so that the inside of the screen covered with red, green, blue phosphorescent material to produce color. The representation of this color is called the RGB color space representation (it is also the most used color space representation method in Multimedia computer technology).
According to the three-color principle, any kind of shade F can be mixed with different components of the R, G, b three-color addition.

F = R [r] + G [g] + b [b]

Wherein, R, G, B are three primary colors to participate in the mixing coefficient. When the tri-primary component is 0 (weakest), the mixture is black, and when the Tri-primary component is K (strongest), it is blended into white light. Adjust the values of the R, G, and b three coefficients to mix a variety of shade between black and white light.
So, where does YUV come from? In modern color TV system, usually use three-tube color camera or color CCD camera to camera, and then the color image of the captured signal by color separation, amplification and correction to get RGB, and then through the matrix transformation circuit to obtain the luminance signal Y and two chromatic aberration signal r-y (that is U), b-y (that is, V), The final transmitter encodes three signals of brightness and chromatic aberration, which are sent out using the same channel. This color is represented by the so-called YUV color space representation.
The importance of using YUV color space is that its luminance signal y and chroma signal u, v are separated. If only the Y signal component does not have a U, v component, then the image represented here is a black-and-white grayscale image. Color TV using YUV space is precisely to use brightness signal y to solve the compatibility problem between color TV and black and white TV, so that black and white machine can also receive color TV signal.
The formula for conversion between YUV and RGB is as follows (RGB values range from 0-255):

Y = 0.299R + 0.587G + 0.114B
U = -0.147r-0.289g + 0.436B
V = 0.615r-0.515g-0.100b

R = Y + 1.14V
G = y-0.39u-0.58v
B = Y + 2.03U

In DirectShow, the common RGB format is RGB1, RGB4, RGB8, RGB565, RGB555, RGB24, RGB32, ARGB32, etc. common YUV formats are YUY2, YUYV, Yvyu, UYVY, AYUV, y41p, Y411, Y211, IF09, IYUV, YV12, YVU9, YUV411, YUV420, etc. As an auxiliary description type (subtype) of the video media type, their corresponding GUID is shown in table 2.3.

Table 2.3 Common RGB and YUV formats

GUID Format Description
MEDIASUBTYPE_RGB1 2 colors, each pixel is represented by 1 bits, need palette
MEDIASUBTYPE_RGB4 16 colors, each pixel is represented by 4 bits, need palette
MEDIASUBTYPE_RGB8 256 colors, each pixel is represented by 8 bits, need palette
mediasubtype_rgb565 each pixel is represented by 16 bits, the RGB component uses 5-bit, 6-bit, 5-bit
mediasubtype_rgb555 each pixel is represented by 16 bits, and the RGB component uses 5 bits (the remaining 1 bits are not used)
Mediasubtype_rgb24 each pixel is represented by 24 bits, and the RGB component uses 8 bits each
Mediasubtype_rgb32 each pixel is represented by 32 bits, the RGB component uses 8 bits (the remaining 8 bits are not used)
Mediasubtype_argb32 each pixel is represented by 32 bits, the RGB components each use 8 bits (the remaining 8 bits are used to represent the alpha channel value)
MEDIASUBTYPE_YUY2 YUY2 format, packaged in 4:2:2 manner
MEDIASUBTYPE_YUYV yuyv Format (the actual format is the same as YUY2)
Mediasubtype_yvyu yvyu format, packaged in 4:2:2 manner
Mediasubtype_uyvy UYVY format, packaged in 4:2:2 manner
MEDIASUBTYPE_AYUV 4:4:4 YUV format with alpha channel
MEDIASUBTYPE_Y41P y41p format, packaged in 4:1:1 manner
mediasubtype_y411 Y411 Format (the actual format is the same as Y41P)
mediasubtype_y211 Y211 Format
MEDIASUBTYPE_IF09 IF09 Format
MEDIASUBTYPE_IYUV IYUV Format
MEDIASUBTYPE_YV12 YV12 Format
MEDIASUBTYPE_YVU9 YVU9 Format

The following is a description of each RGB format.

¨RGB1, RGB4, RGB8 are both color palette-type RGB formats, and when describing the format details of these media types, a palette (defining a series of colors) is typically followed by the Bitmapinfoheader data structure. Their image data is not a true color value, but rather the index of the current pixel color value in the palette. Take RGB1 (2-color bitmap) as an example, for example, the two color values defined in its palette are 0x000000 (black) and 0xFFFFFF (white), then the image data 001101010111 ... (1 bits per pixel) indicates the color of the corresponding pixels: black black and white black white

The ¨rgb565 uses a 16-bit representation of a pixel, 5 bits in 16 bits for R, 6 bits for g, and 5 bits for B. The program typically uses a single word (word, a word equal to two bytes) to manipulate a pixel. When a pixel is read, the meanings of each bit of the word are as follows:
High byte low byte
R-R r r R G G G G G G B b b B. b b
You can use the mask and shift operations together to get the values of the RGB components:

#define Rgb565_mask_red 0xf800
#define Rgb565_mask_green 0X07E0
#define Rgb565_mask_blue 0x001f
R = (Wpixel & rgb565_mask_red) >> 11; Value Range 0-31
G = (Wpixel & rgb565_mask_green) >> 5; Value Range 0-63
B = Wpixel & rgb565_mask_blue; Value Range 0-31

¨RGB555 is another 16-bit RGB format, and the RGB components are represented by 5 bits (the remaining 1 bits are not used). After reading a single pixel with one word, the meanings of each bit of the word are as follows:
High byte low byte
X r r F R R G G G G G B b b B. b b (x means no, can be omitted)
You can use the mask and shift operations together to get the values of the RGB components:

#define Rgb555_mask_red 0X7C00
#define Rgb555_mask_green 0X03E0
#define Rgb555_mask_blue 0x001f
R = (Wpixel & rgb555_mask_red) >> 10; Value Range 0-31
G = (Wpixel & rgb555_mask_green) >> 5; Value Range 0-31
B = Wpixel & rgb555_mask_blue; Value Range 0-31

¨rgb24 uses 24 bits to represent a pixel, and the RGB component is represented by 8 bits, with a value range of 0-255. Note in memory that the RGB components are arranged in the order of: BGR BGR BGR .... You can usually use the Rgbtriple data structure to manipulate a pixel, which is defined as:

typedef struct TAGRGBTRIPLE {
BYTE Rgbtblue; Blue Component
BYTE Rgbtgreen; Green component
BYTE rgbtred; Red component
} rgbtriple;

The ¨RGB32 uses 32 bits to represent a pixel, the RGB components are 8 bits each, and the remaining 8 bits are used as alpha channels or not. (ARGB32 is a RGB32 with an alpha channel.) Note that in-memory RGB components are arranged in the order of: BGRA BGRA BGRA .... You can usually use the Rgbquad data structure to manipulate a pixel, which is defined as:

typedef struct TAGRGBQUAD {
BYTE Rgbblue; Blue Component
BYTE Rgbgreen; Green component
BYTE rgbred; Red component
BYTE rgbreserved; Reserved bytes (used as alpha channel or ignored)
} Rgbquad;

Various YUV formats are described below. The YUV format typically has two main classes: the packaged (packed) format and the planar format. The former stores YUV components in the same array, usually several adjacent pixels to form a macro pixel (Macro-pixel), and the latter uses three arrays to store the YUV three components separately, as if it were a three-dimensional plane. The YUY2 to Y211 in table 2.3 are packaged formats, and IF09 to YVU9 are flat formats. (Note: In the introduction of various specific formats, the YUV components will have subscript, such as Y0, U0, V0 represents the YUV component of the first pixel, Y1, U1, V1 represents the YUV component of the second pixel, and so on. ）

The ¨yuy2 (and YUYV) format retains the Y component for each pixel, while the UV component is sampled every two pixels in the horizontal direction. A macro pixel is 4 bytes, which actually represents 2 pixels. (4:2:2 means that there are 4 y components, 2 U-components, and 2 v components in a macro pixel.) The YUV components in the image data are arranged in the following order:
Y0 U0 Y1 V0 Y2 U2 Y3 V2 ...

The ¨yvyu format is similar to YUY2, except that the order of YUV components in the image data is different:
Y0 V0 Y1 U0 Y2 V2 Y3 U2 ...

The ¨uyvy format is similar to YUY2, except that the order of YUV components in the image data is different:
U0 Y0 V0 Y1 U2 Y2 V2 Y3 ...

The ¨AYUV format has an alpha channel, and the YUV component is extracted for each pixel, and the image Data format is as follows:
A0 Y0 U0 V0 A1 Y1 U1 V1 ...

The ¨y41p (and Y411) format retains the Y component for each pixel, while the UV component is sampled every 4 pixels in the horizontal direction. A macro pixel is 12 bytes, which actually represents 8 pixels. The YUV components in the image data are arranged in the following order:
U0 Y0 V0 Y1 U4 Y2 V4 Y3 Y4 Y5 Y6 Y8 ...

The ¨y211 format samples the Y component in a horizontal direction every 2 pixels, and the UV component is sampled every 4 pixels. A macro pixel is 4 bytes, which actually represents 4 pixels. The YUV components in the image data are arranged in the following order:
Y0 U0 Y2 V0 Y4 U4 Y6 V4 ...

The ¨YVU9 format extracts the y component for each pixel, and when the UV component is extracted, the image is first divided into several 4 x 4 macro blocks, and then each macro block extracts a U component and a V component. When the image data is stored, the first is the Y-component array of the entire image, followed by the U-component array, and the V-component array. The IF09 format is similar to YVU9.

The ¨IYUV format extracts the y component for each pixel, and when the UV component is extracted, the image is first divided into several 2 x 2 macro blocks, and then each macro block extracts a U component and a V component. The YV12 format is similar to IYUV.

¨yuv411, YUV420 format is found in DV data, the former is used in NTSC system, the latter is used in PAL system. The YUV411 extracts the Y component for each pixel, and the UV component is sampled every 4 pixels in a horizontal direction. The YUV420 is not a V-component sample of 0, but instead compares the YUV411 to increase the chromatic aberration sampling frequency in the horizontal direction, reducing half of the chromatic aberration in the vertical direction by the u/v interval, as shown in Figure 2.12.






Color problems:
When we are dvdrip or embedded, we usually encounter some color terms, such as YUV, RGB, YV12, 4:2:2, 4:2:0, and so on. Many people just come into contact with these things, will feel dizzy, unintelligible.
Again, many articles emphasize the film in the process of VDM to choose fast recompress, but fast recompress, Normal recompress, full processing mode what is the difference between.
This article comes in one by one to answer these questions for you.
This article is a summary of the article, so a lot of paragraphs are directly excerpted from other articles. Thank the original author here. This article refers to the Chroma upsampling errors (color upsampling error) and silky articles that were originally contained in DVD benchmark by Don Munsil & Stacey Spears original.

1. What is RGB.
RGB is the meaning of the three primary colors of red, green and blue, r=red, G=green, B=blue.

2. What is YUV/YCBCR/YPBPR.
The luminance signal is often called Y, and the chroma signal is made up of two independent signals. Depending on the color system and format, the two chroma signals are often referred to as U and V or PB and PR or CB and CR. These are generated by different coding formats, but in fact, they are basically the same concept. On DVDs, chroma signals are stored as CB and CR (c for color, b for blue, and R for Red).

3. What is 4:4:4, 4:2:2, 4:2:0.
In the last 10 years, video engineers have found that human eyes are less sensitive to chroma than to brightness. In physiology, there is a law, that is, the retinal stem cells on the human retina more than the retinal cone cells, said more popular, the role of retinal stem cells is to recognize the brightness, and the role of the retinal cone cell is to identify the color. So, your eyes are more discerning about the light and the dark than the color. Because of this, in our video storage, it is not necessary to store all the color signals. Since the eyes are invisible, why waste storage space (or money) to store them?
Consumer videotapes like beta or VHS benefit from leaving more bandwidth on the videotape to the black-white signal (known as "brightness") and leaving a little bit of bandwidth to the color signal (called "Chroma").
In MPEG2 (which is the compressed format used by DVDs), Y, Cb, and CR signals are stored separately (this is why the component video transmission requires three cables). Where the y signal is a black and white signal and is stored at full resolution. However, because the human eye is less sensitive to color information, chroma signals are not stored in full resolution.
The highest resolution format of Chroma signal is 4:4:4, that is, every 4 points y sampling, there is a corresponding 4 points CB and 4 point Cr. In other words, in this format, the resolution of the chroma signal is the same as the resolution of the luminance signal. This format is mainly used in the internal video processing equipment, to avoid the picture quality in the process of reducing. When the image is stored in master Tape, such as D1 or D5, the color signal is usually cut to 4:2:2.
[Center]
In figure one, you can see the luminance, chroma sampling distributions in the 4:4:4 format. As indicated in the figure, each pixel in the screen has a corresponding chroma and luminance sampling information. [/center]
Next is 4:2:2, that is, every 4 points y sampling, there are 2 points CB and 2 points CR. In this format, the chroma signal has the same number of scan lines as the luminance signal, but the Chroma sample count on each scan line is only half the luminance signal. When the 4:2:2 signal is decoded, the "missing" chroma sampling is usually supplemented by a certain interpolation algorithm through the chromaticity information on both sides.
[Center]
Figure two shows the distribution of luminance and chroma sampling in 4:2:2 format. Here, each pixel has a corresponding luminance sample, while half of the Chroma sample is discarded, so we see that the Chroma sample signal has one at every other sample point. In the presence of a picture, the missing chroma information is calculated by interpolating the color of the sides by interpolation. As mentioned above, the human eye is less sensitive to chroma than brightness, and most people can't tell the difference between the 4:2:2 and 4:4:4 colors. [/center]
The lowest resolution format of the Chroma signal, which is the format used by the DVD, is 4:2:0. In fact 4:2:0 is a confusing term, literally understanding that 4:2:0 should be every 4 points y sampling, there are 2 points CB and 0 point CR, but in fact it is not the case at all. In fact, 4:2:0 means that chroma sampling is only half the brightness sample on each transverse scan line, and the number of lines on the scan line is only half the brightness. In other words, both horizontal and vertical, the chroma signal has a resolution of only half the luminance signal. For example, if the size of the whole picture is 720*480, then the luminance signal is 720*480 and the chroma signal is only 360*240. In the 4:2:0, the "missing" chroma sampling is not only to be supplemented by the interpolation complement in the left and right adjacent samples, but also through the interpolation of the chromaticity samples of the whole line. The reason for this is to make the most cost-effective use of the DVD's storage space. Admittedly, 4:4:4 works great, but if we're going to use 4:4:4 to store a movie, we'll have a DVD disc at least two feet (60 centimeters) in diameter.
[Center]
Figure three shows the arrangement of luminance and chroma sampled signals in the 4:2:0 color format non-interlaced picture. As with the 4:2:2 format, only half of the chroma sampling information is in each scan line. Unlike 4:2:2, which is not only the horizontal color information is "thrown" half, the vertical color information is also "thrown" half, the entire screen chroma sampling only One-fourth of the luminance sample. Note that in the 4:2:0 color format, chroma sampling is placed in the middle of two scan lines. Why is that so? It is simple: the color sample on the DVD disk is "averaged" by the color information of the two scanned lines. For example, in Figure three, the first line of color sampling (the row between lines 1 and Line 2) is obtained by the "average" lines 1 and Line 2, and the second row of color samples (the row between line 3 and lines 4) is the same, as is the same as that obtained from the rows 3 and 4.
Although the concept of "average" has been mentioned many times in the article, this "average" is not the average of our usual sense (a+b)/2. Color processing has an extremely complex algorithm to ensure that it minimizes distortion and is close to the original quality. [/center]

4. What is YV12 and what is YUY2.
On the personal computer, these YUV read out will be packaged in some format, for software or hardware processing. Packaging is divided into two types, one is packed format, and the Y and the corresponding UV package together. The other is planar format, which wraps Y and U and v three separately and splits it into three plane (planar).
Both YV12 and YUY2 are a YUV packaging format, and both are packed format. (In fact, only YUY2 is packed format, and YV12 belongs to planar format.) ）
The difference between YV12 and YUY2 is that YV12 is the yuv4:2:0 format, which is the original format stored on the DVD/VCD. YUY2 is the yuv4:2:2 format.

5. Why the film should be selected fast recompress in the process of VDM processing.
The reason for choosing fast recompress now starts with AviSynth 2.5.
The biggest feature of AviSynth 2.5 is that it supports YV12 direct processing. We know that the original MPEG data is yuv4:2:0, that is, the YV12 format, we used to do divx/xvid compression, the process is:
DVD/VCD (YUV 4:2:0), Dvd2avi (YUV 4:2:0->yuv4:2:2->yuv4:4:4, RGB24), Vfapi (RGB24), Tmpgenc/aviut L/virtualdub (RGB24), Divx/xvid Codec (RGB24->yuv4:2:0), MPEG-4 (YUV 4:2:0)
PS. VFAPI internal data can only be transmitted in RGB24, so it will be converted to RGB24 output
Or
DVD/VCD (YUV 4:2:0), Mpg2dec. DLL (YUV 4:2:0->yuv4:2:2), AviSynth 2.0.x (only with filter support Yuv4:2:2, not RGB24/32 filter), VirtualDub (YUV 4:2:2, cannot use VD filter, because VD Filetr are processed on the RGB32, compression to choose Fast recompress, will be directly intact yuv4:2:2, that is YUY2 data to Codec compression)--Divx/xvid Codec (YUV 4:2:2->yuv4:2:0), MPEG-4 (YUV 4:2:0)
So in the middle of the previous processing process to pass several times yuv<-> RGB conversion. This conversion is lossy, and the more you do it, the more severe the original color information loses. And the calculation of this conversion is time-consuming (this explains why we will have more cards when we convert YV12 to RGB output, but the quality of RGB is really much higher). Then someone (Marc FD) thought, anyway the final turn into MPEG to save into yuv4:2:0 format, then why not simply all the way to the end, the whole process to YV12 processing, that is, all the filter is changed to write YV12 version, directly on the YV12 do adjust color, filter noise, IVTC and other work, such as:
1. The amount of data processed is small. (YV12 data, UV than YUY2 less than half, less than RGB 24/32 more)
2. No conversion calculations
So the speed is fast. Plus can avoid yuv<-> RGB conversion loss, it is not double benefit.
So the AviSynth 2.5 that supported YV12 was born.
However, the current VirtualDub still does not support YV12, even if the choice of Fast RECOMPRESS,VD will YV12 input into YUY2. So to get the full YV12 treatment of the benefits, must use VIRTUALDUBMOD, this revision has to support YV12. Only when the fast recompress is selected, VDM will not do any processing, the data will be dropped directly to the encoder compression, so that the YV12 can be retained to achieve the full YV12.