YUV data format Analysis

To collect and process videos, you must learn to analyze YUV data. From the collection point of view, generally the video capture chip output streams are generally in the form of YUV data streams, while video processing (such as H. 264. MPEG Video Codec) is also used for encoding and parsing the original YUV code stream. Therefore, it is important for people who are engaged in the video field to understand how to analyze the YUV data stream. This article briefly introduces how to analyze YUV data streams based on my learning and understanding.

YUV is divided into three components. "Y" indicates the brightness (luminance or Luma), that is, the gray value; "u" and "V" represent chrominance or chroma, which describes the image color and saturation and is used to specify the pixel color.

Like our well-known RGB, YUV is also a color encoding method used in television systems and analog video fields. It separates brightness information (y) from color information (UV, without UV information, you can display complete images, but they are black and white. This design is a good solution to the compatibility between color TV sets and black and white TV sets. In addition, YUV does not require three independent video signals to be transmitted at the same time as RGB does. Therefore, YUV transmission occupies a very small bandwidth.

Now let's get down to the truth and talk about how to analyze the YUV code stream. The YUV code stream has many different formats. To analyze the YUV code stream, you must identify which format you are facing and determine the YUV sampling and distribution of the format. Below I will introduce several common YUV code stream formats for your reference.

1. Sampling Method

The storage format of the YUV code stream is closely related to the sampling method. There are three mainstream Sampling Methods: yuv4: 4: 4, yuv4: 2: 2, yuv4: 2: 0, for details about its principles, you can refer to other articles on the Internet. Here I want to emphasize how to restore the YUV value of each pixel from the code stream based on its sampling format, only by correctly restoring the YUV value of each pixel can the YUV and RGB conversion formulas be used to extract the RGB value of each pixel and then display it.

Use three graphs to visually represent the collection method. Use black spots to represent the Y component of the pixel, and use a hollow circle to represent the UV component of the pixel.

Remember the following section before extracting the YUV component of each pixel.

 
 

  
  
YUV sampling, each y corresponds to a set of UV components.
  
  
 
  
  
YUV sampling, each two y share a set of UV components.
  
  


  
  
YUV sampling, every four y share a set of UV components.
 
 

2. Storage Method

Below I will use a diagram to provide a common YUV code stream storage method, and the storage method is followed by a method to sample the YUV data of each pixel. Among them, CB and Cr are equivalent to u and v.

(1) yuvy format (yuv422)

Yuyv is one of the storage formats of yuv422 sampling. The adjacent Y shares two adjacent CB and Cr. For the analysis of pixels '00 and y' 01, the CB and Cr values are cb00 and cr00, And the YUV values of other pixels are the same.

(2) uyvy format (yuv422)

The uyvy format is also one of the storage formats of the yuv422 sample, except that the UV order is different from that of yuyv. The method for restoring the YUV value of each pixel is the same as that above.

(3) yuv422p (yuv422)

Yuv422p is also a type of yuv422, which is a plane mode, that is, the packaging mode. Instead of storing YUV data in a staggered manner, yuv422p first stores all Y components, then all the U (CB) components are stored, and all the V (CR) components are stored, as shown in. The YUV value extraction method for each pixel is also the most basic extraction method following the yuv422 format, that is, two y share one UV. For example, for pixels '00 and y' 01, the CB and Cr values are cb00 and cr00.

(4) yv12, yu12 format (belongs to yuv420)

Yu12 and yv12 are in the yuv420 format. They are also in plane mode. They package the Y, U, and V components and store them in sequence. The YUV data extraction of each pixel follows the yuv420 format, that is, four Y components share a set of Uvs. Note: In, y '00, y '01, y '10, and y '11 share cr00, cb00, and so on.

(5) nv12 and nv21 (yuv420)

Nv12 and nv21 are in the yuv420 format. They are in the Two-plane mode, that is, Y and UV are divided into two plane, but UV (cbcr) is a staggered storage, instead of being divided into three plane. The extraction method is similar to the previous one, that is, y '00, y '01, y '10, and y '11 share cr00 and cb00.

3. Format analysis and conversion

Yuv420 planar data, taking the 720x488 Size Image yuv420 planar as an example, the storage format is: the total size is (720x480X3> 1) bytes, divided into three parts: y, U, and Vy: (720x480) bytes

U (CB) Weight: (720x480> 2) bytes

V (CR) Weight: (720x480> 2) bytes

The three parts are row-first stored, and the three parts are respectively y, U, and V sequential stored. That is, the YUV data 0--720 × 480 bytes is the Y component value, and the 720 × 480 -- 720 × 480 × 5/4 bytes is the U component, 720 × 480 × 5/4 -- 720 × 480 × 3/2 bytes are the V component.

4: 2: 2 and conversion:

The simplest method:

Yuv4: 2: 2 ---> yuv4: 2: 0 y remains unchanged, and the u and v signal values are sampled in the same line (vertical direction. Yuv4: 2: 0 ---> yuv4: 2: 2 Y unchanged. copy each line of the u and v signal values to form two consecutive rows of data.

In yuv420, a pixel corresponds to a Y, and a 4x4 small square corresponds to a U and a v. For all yuv420 images, their y values are arranged exactly the same, because only y images are gray images. The data formats of yuv420sp and yuv420p are different in principle. P It stores u before V, that is, UV is continuous. The 420sp is stored alternately by UV and UV. (See) with the above theory, I can accurately calculate the size of a yuv420 stored in the memory. Width * hight = Y (SUM) u = y/4 V = y/
4, so the length of yuv420 data in the memory:

Width * hight * 3/2,

Assume that a YUV image with a resolution of 8x4 is in the following format:

The yuv420sp format is as follows:

The yuv420p data format is as follows:

Algorithm for Rotating 90 degrees:

Public static void rotateyuv240sp (byte [] SRC, byte [] Des, int width, int height)
{
Int wh = width * height;
// Rotate y
Int K = 0;
For (INT I = 0; I <width; I ++ ){
For (Int J = 0; j Des [k] = SRC [width * j + I];
K ++;
}
}

For (INT I = 0; I <width; I + = 2 ){
For (Int J = 0; j Des [k] = SRC [Wh + width * j + I];
Des [k + 1] = SRC [Wh + width * j + I + 1];
K + = 2;
}
}
}

Differences between yv12 and i420: Generally, the video data directly collected is in rgb24 format. The size of a rgb24 frame is size = width × heigth × 3 bit, the size of rgb32 is width × heigth × 4. If it is i420 (that is, the YUV standard format is), the data size is size = width × heigth × 1. 5 bit. After rgb24 data is collected, the data in this format needs to be compressed for the first time. The color space of the image is determined by rgb2yuv. Because the standard YUV () is required for x264 encoding ). However, although yv12 is also (), yv12 and i420 are different. There are some differences in the storage space:

Yv12: brightness (row x column) + U (row x column/4) + V (row x column/4)

I420: brightness (row x column) + V (row x column/4) + U (row x column/4)

It can be seen that yv12 and i420 are basically the same, that is, the UV sequence is different.

Continue to our topic. After the first data compression, rgb24-> YUV (i420 ). In this way, the data volume will be halved. Why? Well, this is too basic, and I will not write more. Similarly, if it is rgb24-> YUV (yv12), it is also halved. However, although both are half, if yv12 is used, the effect will be greatly reduced. Then, the data size will be greatly reduced after the x264 encoding. The encoded data is packaged and transmitted in real time through RTP. After arriving at the destination, extract and decode the data. After decoding, the data is still in YUV format. Therefore, a conversion is required so that the windows driver can process the data, that is, yuv2rgb24.

Yuy2
Yes [y0 U0 Y1 V0]

The differences between yuv420p and yuv420 are different in storage formats: yuv420p: yyyyyyyy uuuuuuuu vvvvv yuv420: YUV

The yuv420p, Y, U, and V components are in flat format, which is divided into i420 and yv12. The i420 format and yv12 format are different in the U plane and V plane. In i420 format, the U plane follows the Y plane, and then the V plane (that is, YUV); but yv12 is the opposite (that is, yvu ). Yuv420sp, Y component plane format, UV packaging format, that is, nv12. Nv12 is similar to nv21, and U and V are arranged in different UV order.
I420: yyyyyyyy uu vv => yuv420p
Yv12: yyyyyyyy vv uu => yuv420p
Nv12: yyyyyyyy uvuv => yuv420sp
Nv21: yyyyyyyy vuvu => yuv420sp

3. Summary

Several common YUV code streams are listed on the list. Before processing the YUV code streams, you must first know which one your code streams belong to and then process them accordingly.

Finally, I would like to answer another question, that is, to understand the YUV code stream format. What can we do? The most common one is to extract all Y components, and then use VC or Matlab to display the gray value (Y component) of the image you have collected, in this way, you can quickly know whether the image you have collected is faulty. Later, I will continue to write some articles about how to extract, convert, and display the original YUV code streams. If you are interested, please stay tuned.