Getting started video collection and processing

To collect and process videos, you must learn to analyze YUV data. From the collection point of view, generally the video capture chip output streams are generally in the form of YUV data streams, while video processing (such as H. 264. MPEG Video Codec) is also used for encoding and parsing the original YUV code stream. Therefore, it is important for people who are engaged in the video field to understand how to analyze the YUV data stream. This article briefly introduces how to analyze YUV data streams based on my learning and understanding.

YUV is divided into three components. "Y" indicates the brightness (luminance or Luma), that is, the gray value; "u" and "V" represent chrominance or chroma, which describes the image color and saturation and is used to specify the pixel color.

Like our well-known RGB, YUV is also a color encoding method used in television systems and analog video fields. It separates brightness information (y) from color information (UV, without UV information, you can display complete images, but they are black and white. This design is a good solution to the compatibility between color TV sets and black and white TV sets. In addition, YUV does not require three independent video signals to be transmitted at the same time as RGB does. Therefore, YUV transmission occupies a very small bandwidth.

Now let's get down to the truth and talk about how to analyze the YUV code stream. The YUV code stream has many different formats. To analyze the YUV code stream, you must identify which format you are facing and determine the YUV sampling and distribution of the format. Below I will introduce several common YUV code stream formats for your reference.

1. Sampling Method

The storage format of the YUV code stream is closely related to the sampling method. There are three mainstream Sampling Methods: yuv4: 4: 4, yuv4: 2: 2, yuv4: 2: 0, for details about its principles, you can refer to other articles on the Internet. Here I want to emphasize how to restore the YUV value of each pixel from the code stream based on its sampling format, only by correctly restoring the YUV value of each pixel can the YUV and RGB conversion formulas be used to extract the RGB value of each pixel and then display it.

Use three graphs to visually represent the collection method. Use black spots to represent the Y component of the pixel, and use a hollow circle to represent the UV component of the pixel.

Remember the following section before extracting the YUV component of each pixel.

 
 

  
  
YUV sampling, each y corresponds to a set of UV components.
  
  
 
  
  
YUV sampling, each two y share a set of UV components.
  
  

  
  
YUV sampling, every four y share a set of UV components.
 
 

2. Storage Method

Below I will use a diagram to provide a common YUV code stream storage method, and the storage method is followed by a method to sample the YUV data of each pixel. Among them, CB and Cr are equivalent to u and v.

(1) yuvy format (yuv422)

Yuyv is one of the storage formats of yuv422 sampling. The adjacent Y shares two adjacent CB and Cr. For the analysis of pixels '00 and y' 01, the CB and Cr values are cb00 and cr00, And the YUV values of other pixels are the same.

(2) uyvy format (yuv422)

The uyvy format is also one of the storage formats of the yuv422 sample, except that the UV order is different from that of yuyv. The method for restoring the YUV value of each pixel is the same as that above.

(3) yuv422p (yuv422)

Yuv422p is also a type of yuv422, which is a plane mode, that is, the packaging mode. Instead of storing YUV data in a staggered manner, yuv422p first stores all Y components, then all the U (CB) components are stored, and all the V (CR) components are stored, as shown in. The YUV value extraction method for each pixel is also the most basic extraction method following the yuv422 format, that is, two y share one UV. For example, for pixels '00 and y' 01, the CB and Cr values are cb00 and cr00.

(4) yv12, yu12 format (belongs to yuv420)

Yu12 and yv12 are in the yuv420 format. They are also in plane mode. They package the Y, U, and V components and store them in sequence. The YUV data extraction of each pixel follows the yuv420 format, that is, four Y components share a set of Uvs. Note: In, y '00, y '01, y '10, and y '11 share cr00, cb00, and so on.

(5) nv12 and nv21 (yuv420)

Nv12 and nv21 are in the yuv420 format. They are in the Two-plane mode, that is, Y and UV are divided into two plane, but UV (cbcr) is a staggered storage, instead of being divided into three plane. The extraction method is similar to the previous one, that is, y '00, y '01, y '10, and y '11 share cr00 and cb00.

3. Summary

Several common YUV code streams are listed on the list. Before processing the YUV code streams, you must first know which one your code streams belong to and then process them accordingly.

Finally, I would like to answer another question, that is, to understand the YUV code stream format. What can we do? The most common one is to extract all Y components, and then use VC or Matlab to display the gray value (Y component) of the image you have collected, in this way, you can quickly know whether the image you have collected is faulty. Later, I will continue to write some articles about how to extract, convert, and display the original YUV code streams. If you are interested, please stay tuned.