Basic audio and video knowledge
1. Basic concepts of video
RGB and YUV
RGB refers to red, green, and blue, which is widely used, such as display and pixel values in BMP file formats. YUV mainly refers to the brightness and two chromatic aberration signals, the conversion relationships between luminance and chrominance can be checked by ourselves. Our videos are basically in YUV format.
YUV format
YUV file formats are divided into many other formats. If the storage format is used, more formats are available, such as yuv444, yuv422, yuv411, and yuv420. Video Compression uses the 420 format, this is because the human eyes are more sensitive to brightness and have a relatively poor color. In addition, pay attention to the meanings of several English words, such as packet, planar, interlace, and sive ssive.
Frame Rate
The refresh speed of the image per second. For pal TVs, the frame rate is 25 frames per second, and for NTSC TVs, the frame rate is 29.97 frames per second. Our commonly used computers also have a refresh rate. Generally, the refresh rate of computers should be more than 75Hz, so the human eyes will not feel the flash.
Interlace and line-by-line scanning)
Generally, the TV is scanned by line, while the display is scanned by line. Here is the concept of a field. The line-by-line scan is equivalent to one frame or two, while the line-by-line scan is equivalent to one frame.
Bit Rate
Its unit is bit per second. Generally, all the bandwidth descriptions are bit. The unit that describes the storage capacity is generally large B, that is, byte (bytes ).
Resolution
The resolution of an image refers to the number of shards. Generally, the most commonly used is CIF, that is, 352*288. 4cif naturally means 704*576, the resolution of D1 is strictly 720*576, which is about the same as that of 4cif. Of course, there are still many high-definition resolutions. I don't know much about these. If you are interested, please check them. In addition, in many foreign countries, the CIF height is 240, because their frame rate is higher than ours (29.97Hz). Naturally, the height is smaller.
Real-time and non-real-time
It is mainly used to describe the encoder. It has two meanings: one is to ensure the frame rate, that is, 25 frames per second, and the other is "live", which means live broadcasting, the so-called "live broadcasting ".
Latency
It is also an important indicator to describe the encoder. Generally, people from 500 ms to Ms will not feel very obvious. By ms, we can still clearly feel it.
Audio/Video Synchronization
As an application of video conferencing, lip synchronization is generally required ". The basic method to ensure audio and video synchronization is the time stamp ).
Compound video and S-video
NTSC and PAL Color video signals are like this-First there is a basic black and white video signal, and then a color pulse and a brightness signal are added after each horizontal synchronous pulse. Because color signals are "superimposed" by multiple types of data, they are called "Compound videos ". S-video is a video interface with higher signal quality. It eliminates the signal superposition method and effectively avoids unnecessary quality losses. Its function is to separate RGB primary colors from brightness.
NTSC, pal, and SECAM
A baseband video is a simple analog signal consisting of analog video data and video synchronization data. It is used to display images correctly at the receiving end. The signal details depend on the Applied video standards or "standards"-NTSC (National Television Standards Committee, National Television Standards Committee), Pal (line-by-line inverted phase, phase alternate line) and SECAM (sequence transmission and storage of color television systems, a French television system, sequential Couleur avec memoire ).
China's television signals are generally pal, while the US and Japan are NTSC. The frame rates and image sizes of these two formats are different.
Number of lines
When we buy a camera, we often mention the concept of a number of lines, which is actually the height of the resolution ). For example, for a pal D1 image, the number of lines is 576.
Brightness, saturation, and contrast
The English names are brightness, saturation, and contrast. This is an important indicator of three images.
2. Basic concepts of audio
Sampling Rate
The audio frequency sampling rate is similar to the video frame rate, meaning the number of samples per second. The sampling rate of g.711 is 8 K (the human voice is probably within this frequency range ), the typical samplerate supported by MP3 is 44.1 kHz (more than twice the response frequency of human ears ). Obviously, the original MP3 compressed sound is much better than g.711.
Sampling Accuracy
It is the quantization coefficient of each sample for modulus conversion. G.711 is an 8-bit sampling precision, while MP3 is typically 16bit.
Echo Elimination
The biggest audio problem in video conferencing applications. The reason for Echo generation is complex. Generally, there are three sources of latency for voice transmission over the Internet: compression latency, grouped transmission latency, and processing latency. Voice compression latency is the main latency for Echo generation. For example, in the g.723.1 standard, the maximum latency for compressing a frame (30 ms) is 37.5 Ms. The group transmission latency is also an important source. The test shows that the maximum transmission latency from the end to the end can be over Ms. Processing latency refers to the encapsulation latency and buffer latency of the voice packet.
Basic audio and video knowledge
