"Deep decomposition" listening to the fun Pat cloud product Manager Anatomy Video Basics (2)

" with the continuous progress of technology, the production and processing of video technologies gradually reduced the threshold, increasing information resources, while the video information content more abundant and complete congenital advantages, in recent years has gradually become the mainstream." In the basic knowledge (1), we have already talked about the video technology of the analog era and the digital age. Next, the video coding and compression, picture compression, motion compression, the advent of Internet video applications to do a detailed introduction. "

Video encoding and compression

Video coding and compression, is a very important technology of digital video, so that it directly affects the video in various fields of application. Without the continuous improvement of video coding technology, it is not possible today to enjoy the convenience of video in every aspect.

First, video coding is a very complex project, far more difficult to compress audio and image. Secondly, video coding is a multistage compression process, not a single compression scheme. Of course, if it were not for such a complicated project, the video files were much larger than we thought. Let's give an example:

According to the CCIR 601 video signal acquisition standard, a standard PAL TV signal is converted into a digital signal, according to the common non-professional-grade sampling standard 4:2:0 (you want to support higher also not ah, civil-grade equipment do not get higher sampling rate), The resulting digital file for the video content generated per second is 21MB. So how big is the 1-minute video file? 1260MB that big.

Then, if according to the RGB color expression, 720x576 resolution, each sample point 3 base color, each base color is 8bit data, 25 frames per second screen. After a simple calculation, we can easily conclude that the result is 720x576x3x8x25=237.3mbit=29.67mbyte. Then the 1-minute video is 1780MB. I'd like to see a 1-minute video from a user who doesn't have a lot of files to generate. This is only the standard definition, if it is high-definition 1080P, it is 69.5tb!

As can be seen from the above example, even if the video is not compressed, using YUV color to store information, compared to the use of RGB color to store information, the capacity is still smaller. So it can also be said that YUV color is the first-level compression method of video coding.

It needs to be interspersed with a topic about color scale.

The meaning of the color scale is the number of transition steps in the middle when the color is from the maximum. If the brightness of the black and white signal, the level of 2 o'clock, then it is only two colors, all white and All black. If it becomes level 256 (the more common color scale standard), the result is the following:

In the same RGB three colors, each color is a colored order. 8bit data can store 256 levels, then the RGB tri-color can achieve 16.77 million colors, that is, 24-bit color.

Note: The computer color system has 32-bit color, in fact, 24-bit color outside the addition of a 8-bit alpha transparent layer, so also known as RGBA.

Can you use a higher color scale? More than level 256 color scale OK? Of course, but the general display is not supported. But it does have a high-color display, the highest level of display can support 10bit color information, that is, 1024 levels of color scale. Of course the price is unthinkable!

EIZO GX540 Medical Display, level 1024 color scale monochrome display

The price is unclear, but not less than 100,000 yuan

SONY bvm-x300 Master Monitor, OLED display 10bit color

Can be displayed in 1 billion different colors

The price, I remember, is probably more than 360,000.

Also have to tell you a bad news, the general civilian low-end display using the TN-type LCD panel, are 6bit, that is, RGB each color only 64 levels, altogether can display color only 260,000 kinds. Of course, you can choose 32-bit color mode, but its 16.77 million colors, is interpolated by the value of the conversion, not really 16.77 million colors. Real support for 16.77 million colors of the display, in fact, is not very cheap.

Picture compression

If each frame of the video screen, according to the RGB color saved, the file will be very large. For example, the PAL-made video screen produces files with 1.2MB.

If you compress the video footage for each frame, you can significantly reduce the file size of your video. And the most common image compression algorithm we know is JPEG. JPEG is the abbreviation for Joint Photographic Experts Group (joint Image Expert Group) and is the first international image compression standard.

First, JPEG compression is the YUV color component of the image encoded separately, the main algorithm used is DCT (DCT for discrete cosine Transform, discrete cosine transform). It is a transformation related to the Fourier transform, which resembles a discrete Fourier transform (DFT for discrete Fourier Transform), but uses only real numbers. DCT is a very high compression rate of low distortion compression algorithm, you can compress the image to 1/5 to 1/10 size, and the quality of the basic not much change.

Then using the JPEG compression algorithm, the original image size of 1.2MB per frame, now become about 180KB, a lot of reduction. And the video size of every second becomes the 4.4mb,1 minute video is 263MB. Suddenly a lot smaller. The video encoding method using this algorithm is called Motion JPEG, also called MJPEG. Note that video compression also has a more well-known method called MPEG, but not the same as MJPEG, the two are very different.

Motion compression

Although the JPEG algorithm, the video can be made several times smaller, but still relatively large. For the transmission and storage, the threshold is still too high, only suitable for the use of professional institutions such as the radio and television industry.

So what can you do to compress video files even smaller? That is the inter-frame compression method.

When it comes to inter-frame compression, it must be mentioned that an organization, MPEG (Moving Picture Experts Group, motion graphics Expert Group) is an ISO (international standardization Organization, ISO) and the IEC (International Electrotechnical Commission, International Electrotechnical Commission), established in 1988 to develop international standards for motion picture and speech compression organizations.

Now you know, MPEG is actually the name of an organization. Of course, there are many representative compression algorithms in this organization, all of which are named after Mpeg-x. Therefore, we are accustomed to call MPEG compression method.

First of all, motion compression uses the inter-frame compression method. And what is the inter-frame compression method?

Since the video is made up of many frames, the motion is very small in a short interval due to the characteristics of the motion. The other is the movement of the picture, there are many and no movement of the picture information. There is even a lot of frame images that are rarely changed between pictures. It is simply too wasteful to record these unchanged image information in such a repetitive way.

Inter-frame compression is to eliminate as much information as possible of the content that is not changed in the adjacent screen. For example, the picture is a person riding a bicycle, the background is unchanged, and the cyclist from one end of the screen to the other side. So this time, you can put the background part that is not covered, just save one copy on the line. All that remains is to record the entire animated picture of a cyclist.

At that time, the principle is relatively simple, the implementation is more difficult. The concept of keyframes and non-keyframes is used first when compressing between frames. A keyframe is a frame of image that you want to save all the data on the screen, and use this image as a reference. Each frame after the keyframe is followed by a keyframe and a previous frame of the screen, recording where the screen changes, removing the duplicate information.

The early compression algorithm was to adopt such a strategy, such as MPEG-1. Its application products are probably more familiar with--VCD.

Here by the way, the full name of Mp3,mp3 is called MPEG-1 Layer3. This means that the MP3 compression format is a subset of the MPEG-1 compression standard. With MP4 is a completely different concept.

VCD Although in a CD (650MB capacity) can be placed under the length of almost a movie, has been the compression rate is amazing. Of course this is also the premise of sacrificing the screen:

• VCD resolution is very low, only 352x288 (corresponding to the PAL system), than the standard TV screen clarity is much smaller.

• VCD in the case of motion is not too obvious picture quality can also be accepted, if it is moving quickly in the picture, there will be a lot of horrible mosaic.

With VCD products, people are happy. Here can incidentally say, VCD machine is invented by Chinese, that company called Wanyan. However, VCD technology is Philips, SONY, Panasonic, JVC and other companies jointly developed standards, and the production of VCD chip company is the United States C-cube Company. How to say, although the technology is invented by foreigners, but they do not value the technology, so there is no product formation. Instead, the people carried it forward.

At the same time, Europe and the United States in fact, VCD is not too cold. Because they are still in the VHS era (interested students can search for VHS, as well as Sony's Betacam and JVC's VHS standard War), and VCD first not cheap, second quality is not high, the third can not be recorded only play.

In fact, with the VCD products, the world is still very shocked. Everyone thinks it's really convenient to put a movie in a small disc. But VCD quality is really bad, is there any new technology can achieve a smaller capacity of higher definition? That was the later launch of the MPEG-2.

MPEG-2 This standard is the first popular global compression technology, the standard set Time is 1994 (VCD Standard is 1993). Although it has been over more than 20 years, it is still one of the most important video compression formats today. In addition to a large number of DVD products, more important is the current radio and television field of digital TV DVB-T standard, still using the MPEG-2 compression standard (in China).

What is the MPEG-2 relative to MPEG-1?

1. The picture has been greatly improved and more flexible. MPEG-1 almost all applications are concentrated on the VCD, the resolution is very small, and can not be changed. The MPEG-2 can be used for medium definition (D1 standard, PAL or NTSC TV standard) to display high definition video content. This means that even 720P, 1080P and other such high-resolution video, MPEG-2 still applies.

2. Added GOP mode, using IBP frame structure. The original frame compression mode, in the large dynamic scene mosaic is very serious. After the MPEG-2, there was a lot of improvement, because frame B frames were used, forward prediction frames were used, and the compression rate was variable. In general, there is no mosaic in the big dynamic.

3. Added a lot of additional information, more powerful features. For example, to support stronger interaction and command control (we have not think of VCD 2.0 when the screen has a menu can be selected, DVD than this powerful), support the transmission stream form (Ts,transportstream, is can be used for live, also not afraid of file damage on the whole), multi-track and multi-channel.

But MPEG-2 also has the insufficiency, mainly is it is facing the industrialization video information production distribution domain, namely only is suitable for the television station, the DVD publisher, the satellite communication and so on domain, is not suitable for the civil. Because the code flow is really big, bigger than MPEG-1. Although a CD can be installed the next full movie, it is because the capacity of the CD-ROM increased from 650MB to 4.3GB, or even 7.2GB.

Advent of Internet video applications

The early broadband speed only 1~2mbps (56K modem and ISDN era there is no video anything), want to play DVD movie Online is impossible (at least 5~10mbps), VCD is not and technically not supported. The scientists of MPEG organization began to study the video compression method which is suitable for playing on the network, that is, the MPEG-4 compression format which is introduced later.

The obvious feature of MPEG-4 is that it is suitable for playing on the network, more flexible and more powerful:

• The compression ratio is higher and more flexible. MPEG-1 compression ratio is 20~30 times, MPEG-2 compression ratio is 10~20 times, MPEG-4 compression ratio ranging from dozens of to 100 more;

• For picture content, you can use a different compression ratio to use a high compression ratio for non-important objects, and a low compression ratio for important objects. This can be used to ensure that the main picture quality of the compression ratio is higher;

  
  

• Different objects can use different coding algorithms to further improve the compression efficiency;

  
  

• Audio and video collocation more flexible;

• More interactive, especially for the internet model.

MPEG-4 later produced a lot of derivative compression algorithm, the more famous is XviD and Dvix. In fact, MPEG-4 's popularity is inferior to XviD and Dvix, because at that time, MPEG-4 in order to adapt to the Internet lower bandwidth speed, most applications are some low-resolution low-stream video. XviD and Dvix, though derived from the MPEG-4 system, are optimized for video-file archiving, which can be smaller than a few times as large as DVDs, storing video content that is very close to DVD quality. Has been greatly loved by the users, so that at that time has become pirated films must be selected format.

realmedia--once the King

Once the biggest winner of Internet video is the real Network, it is also the first to achieve Internet-based streaming video (online viewing). While still in the 56K modem narrowband period, Real Network has provided video online viewing function. I used to dial the PSTN in the era, experienced through the RealPlayer to watch the NBA live. If the current standard to measure the product of that period, it is the delay is huge (often loading a few minutes), the quality of the terrible (ultra-low resolution, mosaic serious), regular lag.

But with the gradual popularization of broadband, RealMedia's great advantage can be cast. At that time, RealMedia was a worthy king.

First, the RealMedia compression standard is not a MPEG-4-derived version, but a private compression standard. This standard was created by real network and is unique. RealMedia has a great compression ratio, far beyond the MPEG family. For example, a standard DVD format of the movie, about 4.3GB capacity, if the use of Dvix compression, probably can be compressed to 700MB, and the RM format compression, not even half of 700MB. Even to the later RMVB compression format, it is not nearly 500MB.

Second, Real Network provided the world's most comprehensive streaming media system solution at the time, but it was charged. The competitor at that time was Microsoft's Windows Media Encoder, free but not fully functional. RealMedia Encoder offers a complete streaming solution for both VOD and live mode, although it is more expensive (about 10,000 dollars in impressions).

For the commercial applications of enterprises, to do live and on-demand business, natural RealMedia Encoder services will be more reliable. Although charged, the system is stable, reliable, and service-enabled (Windows Media encoder can only ask for help in the MSDN community, without technical support). So the real company was already in the monopoly position at that time.

At that time, however, the biggest problem faced by real was piracy. Because the RealMedia player is free, the encoder, decoder, streaming media server and so on are all charges. As the real format becomes more prevalent, piracy is rampant. Of course, real company is not so deep-pocketed as Microsoft, so naturally to be banned everywhere. This can only be said in the early 21st century of the Internet status quo, if it is now, such as real such a company VCs do not squeeze the door? The company's valuation is less than tens of billions of dollars to say it.

Unfortunately, the closures and charges, combined with the ban, have given the competition a lot of opportunities. Then there is the rise of flash streaming media.

How strong can the RealMedia be? In the 21st century, almost 10 years before 2010, the real format became almost the only choice for the Internet video format. At that time almost all pirated movies and video files were all based on RM and RMVB format. What about 21st century ago? That's Avi's World, Koko.

RealMedia video compression is divided into two stages, the first stage is RM format, the second stage is RMVB. As the RM format, although the compression rate is surprisingly high, but the problem is poor quality, mosaic serious. With the popularity of broadband, the demand for picture quality is increasing, and with the pressure of rivals (Dvix and XviD), RM has been upgraded to RMVB. More out of the "VB" the two subtitles, in fact, refers to the "Variable bitrate" dynamic bitrate or variable code rate. The explanation about the rate of code we do at the end of the article.

RM upgrade to RMVB, has been criticized for the quality of the problem has been improved, but also can support high definition (maximum to 720P). But RMVB's time has been too late for competitors, and the reason for commercialization, as well as the pace of updates is getting slower. Eventually disappears into the vision of the Internet.

The rise of Flash video

Unlike RealMedia's industry-wide popularity, Flash Video (hereinafter referred to as FLV) is mainly used in streaming media, providing VOD and live live streaming services. Together with Dvix and XviD became the gold combination of the second phase.

The compression encoding of FLV is not derived from MPEG-4, but is another powerful standard h.26x system, which first appeared at the Macworld Expo Conference in 1997. It was interesting to say that FLV was born with an Apple and was abandoned by Apple at the end.

First of all, FLV is not a compressed encoding format, but a packet format (such as AVI, MKV, MP4, MOV, these files, are a packet format. As to the problem of packet format, it is not explained in detail because of the deep technology involved. Interested users can know their own, @ I will do OH). FLV uses video compression Coding in fact there are quite a lot of, start is Sorenson video and Sorenson Video Pro, as well as Sonrenson Spark, later joined the Sorenson MPEG 1/2/4, and then is the most popular at present.

In fact, many of the early Sonrenson compression coding formats are derived from QuickTime compression encoding and are also based on h.263 compression coding applications. So we don't introduce the character of this code alone.

As for the performance of FLV in the market, in fact, most people may know better. After all, the domestic video stations, such as Youku Tudou, cool 6, PPS, PPTV, and so on, are all used in the early flv; foreign stations such as YouTube are also the first FLV users. Just as Apple advocated a full-scale campaign to flash, all the video stations have to start to HTML5 transformation.

Windows Media Video

WMV has always been a tepid compression coding format, debut very early, but rarely applied. As early as the RealMedia period, Microsoft has introduced the WMV compression format. You can implement streaming media applications with Windows Media Encoder, or you can store them individually as files. It can be said that the same way as real company, but is free.

In fact, Windows Media Encoder (hereinafter referred to as WME) has not been any improvement, belonging to a long history but no one knows the breed. Early in the competition with the RealMedia in the wind, but at least a lot of users know. Later, after the FLV, the basic can not see the figure of WME.

As for the compression format of WMV, the first is a private format. But at WMV 9.0, Microsoft's standardized approach to the SMPTE Society was incorporated into the VC-1 standard system. That is, other families can also share this coding technology.

In the early WMV standard, such as WMV 7.0, is based on the MPEG-4 PART2 implementation of the coding algorithm. The earliest version has no data, but it can be seen that WMV mainstream version is mainly based on MPEG-4 encoding.

h.26x Family

In addition to the high-profile MPEG organization (which is part of the ISO International Organization for Standardization), there is an organization that has a prominent contribution in the field of video coding compression, which is VCEG (video Coding Experts Group). Vceg belongs to another very powerful organization of the ITU (International Telecommunication Union) subordinate departments, probably most people do not know the name, but you must know that they provide the standard--h.264.

VCEG organization mainly compiles is the h.26x standard system, mainly has the h.261,h.263,h.264.

H.261 mainly in the field of video conferencing, but also the main application in the field of monitoring security. As mentioned earlier, low-resolution low-stream video.

H.263 is H.261 's enhanced version, mainly to support higher resolution (16CIF), using a more advanced motion compensation algorithm. Later upgraded to the h.263+ and h.263++, the algorithm performance and resolution have been significantly improved.

In addition to the high popularity of H. A, there is a h.262 standard, but the application is very little, do not say.

H. MPEG-4 part10 AVC

At present, we are most familiar with the compression encoding format is H. MPEG-4, in fact it has another name, Part10 AVC.

The reason is that the standard is not a set, but two of the world's most authoritative coding experts to do together. That is the VCEG organization under the ITU and the MPEG organization under ISO. If you are interested, you can search the ITU to see how important it is in the world standard system. At present, the world's communication network standards, such as 2G GSM, 3G WCDMA, TD-SCDMA, CDMA2000, and 4G Fdd-lte and Tdd-lte are issued by the ITU.

H.264/mpeg-4 AVC is a fusion of two authoritative organizations of knowledge crystallization, is currently the world's best coding algorithm. It has a lot of features, and I'll just list some of the features that you can perceive. For more details, you can check the information yourself.

• Coding compression rate is high, also very flexible. Under the same picture, the compression rate is twice times of MPEG-2, and the MPEG-4 is 1.5~2 times. It can also be used with very high bitrate (MPEG-2) and very low bitrate (MPEG-2 1/8) for faster transmission and higher picture quality requirements.

• The dynamic effect is even better, basically completely eliminate the mosaic phenomenon. Like the big dynamic scene in the Transformers movie, you can still see the picture very clearly.

• Higher compression efficiency, such as a static screen can achieve ultra-high compression ratio. This is because h.264/mpeg-4 AVC maximizes the removal of redundant data, resulting in improved coding efficiency.

• Error repair ability, can be more efficient transmission in the Environment of Poor network QoS.

• Suitable for a variety of industry applications, whether it is video conferencing, security monitoring of such high compression use, or Internet streaming dynamic network environment use, as well as radio and television, such as the use of high-quality standards. So the result you see is that almost all industries are using this standard (domestic cable TV is still a MPEG-2, mainly due to the high bandwidth of cable television and the fact that it is relatively abundant, plus the high cost of technology upgrade). such as small to CIF size of the video, up to 4K standard movie files, you will find that they are basically based on h.264/mpeg-4 AVC compression.

• Industry wide application also benefit from h.264/mpeg-4 AVC this encoding format product line is sound.

As you may all know, the higher the compression rate (the same picture), the more complex the coding, the higher the computational capacity. The coding and decoding of H. MPEG-2 is much higher than that in the past, such as MPEG-4, and so on. So there will be an earlier computer in the soft decoding (CPU decoding) of the video of the time, easy to appear in Dayton. This is because of the high computational performance requirements.

But fortunately, there are a large number of dedicated codec chips, as well as dedicated to the H. T codec optimized GPU (mobile phone GPU and computer graphics GPU), so that fluency and speed are guaranteed. So you'll find some high-quality videos that may be playing on a computer, but playing on some set-top boxes can be very smooth.

h.265

In fact, the h.265 has not been popular, the standard has been established. The main features are further improvement of compression efficiency, support for UHDTV, better signal-to-noise ratio and so on.

At present, a number of mobile phones, surveillance security equipment, video conferencing equipment began to use h.265 encoding format. It is expected to become more and more popular.

QuickTime Family

Speaking of QuickTime, we will certainly think of Apple company. Yes, QuickTime is Apple's complete set of coding, decoding, playback and streaming solutions. The compact format of QuickTime was private early, and because of the low prevalence of early Mac series computers (which did not have an iphone at that time), there was very little use of QuickTime in this format, including their streaming media applications.

Only later on, with a large number of iphone users, QuickTime is known to all. But this time Apple has already added the QuickTime standard to the MPEG-4 standard system, and later H. As a result, it is clear that Apple is currently using the compression standard of H.

Compression coding Complete

In fact, in addition to the above mentioned mainstream non-mainstream video compression coding format, there are a lot of people do not understand. Because the application is very few, and now almost all of them are unified by the market, so do not introduce. The following table is the current collection of all video compression coding standards:

A detailed description of the code stream

A simple code stream is how much video is generated per second, typically in bps (bit per second). Where B stands for bit (bit) and is easily confused with B. In fact, b stands for byte, meaning "byte". 1Byte bytes =8bit bits. Generally, when you store a file, B is usually used, and when you transfer a file, it is generally expressed as B in order to express speed.

Bitstream is often used to tag video properties when video is transmitted, because streaming media, especially in the live system, does not have much meaning in the size of the file. Because the audience may come in at any time, wait until the end of his watch to get to the size of the video file. Another point is that the TS stream file does not need to be read from the beginning, it does not have the concept of a file tail, and can start reading from any one fragment. So this time the size of the file is not very meaningful.

In order to accurately evaluate the video, it is necessary to use the code stream parameter. Because the bitstream represents the amount of data you need to transfer per second, you need to match your network. If your network bandwidth is 2Mbps, this represents the ideal state. Then if you play a bitrate of 1.5Mbps (approximately equal to 1500Kbps) streaming video, it is likely to be Kaka. The main reason is the network QoS (quality of service, the quality of services) problem, it does not guarantee that data can always be at the highest speed, after all, the network environment is very complex.

In the early days, the compression coding standard used a constant code stream encoding (CBR mode), that is, the video file size produced every second is exactly the same. For example, the MPEG-1 stream is 1.5Mbps, then 60 minutes of video produced by the file is 675MB, just a VCD CD capacity.

When the MPEG-4 era, due to the complexity of the network environment, if the use of constant code flow strategy, when the network speed suddenly become worse, the video is easy to lag. In addition, if there is a large dynamic scene in the video screen, the original code stream may cause the picture quality to become worse, need to temporarily reduce the compression rate. Therefore, according to the application of these two cases, different streams can be taken according to the actual situation when streaming live encoding and video compression coding. Therefore, a variable code stream encoding form (VBR mode) is generated.

In the case of VBR encoding, in order to fully improve the compression coding rate, you can use 2 times compression method, that is, 2pass. However, the 2-time compression method will greatly reduce the speed of compression, although video stream lower picture quality is better, but the encoding time is also greatly elongated.

END

Tips

The old driver takes you to know the common movie file information and characteristics

Now many of the film's files seem to be long, and we don't necessarily understand. For example, the following:

Children.of.man.2006.bdre.1080p.x264.ac3-siluhd.mkv

For such a file name, you need to split it apart to see.

L was definitely the name of the film or TV show at first, but sometimes there are some suffixes to indicate the particularity of the version, such as the director's editor, Unrated, the limited, and the limited projection version.

The second part is the year of release, such as the above 2006;

The third part is the video source, such as the above Bdre, indicating that it is from the Blue disc (BD) was re-encoded. The same approach also means that hdrip from Hd-dvd, HR-HDTV to re-suppress from HD TV signals, and TVRip. In addition, some of the logo for the cam from the movie theater with camera candid, TS, although in the film with camera candid, but audio with the theater to provide audio output so that the audio does not interfere with the cinema, the TC is directly from the film on the autocratic copy (no digital processing, brightness does not work, the quality is very general);

L Part IV is the resolution of the current video, it is obvious that the above 1080P represents the 1920x1080 resolution, progressive scan;

The second part is the current video using the compression format, the above X264 to use the X264 encoder compression, and some XviD and so on are represented by the encoder. What's special is that the Remux represents a re-encapsulation of the video data directly from the HD disc (BD or HD-DVD) (not re-encoded). The highest quality, but the general document will be very large;

L Part VI is the current film audio compression encoding format, common with DTS, AC3, DD5.1, AAC, LPCM, MP3. Of course, some use the next generation HD audio format, such as Dtshd,truehd;

The expression after the horizontal line is the name of the suppressed group.

There is no get new skill! than the heart!

"Deep decomposition" listening to the fun Pat cloud product Manager Anatomy Video Basics (2)