Android Multimedia Framework Summary (v) Multimedia basic concept

Reprint please the head source link and the tail two-dimensional code together reproduced, this article from:
http://blog.csdn.net/hejjunlin/article/details/52431887

The other process of MediaPlayer from prepare to playback is described in the previous article, but many of the basic concepts of audio and video may not be clear enough, and there are often some fundamental concepts for multimedia development today. Look at today's agenda:

• Classify the concept of miscellaneous chaos
• Video section Related
• Audio section Related

First look at a picture, so often say, whether really understand their true meaning:

Classify the concept of miscellaneous chaos

• Video resolution
  
  • SD, HD, 720P ...
  • SD: means "standard definition", which is a video format with a physical resolution of 720p or less. The so-called SD, English is "standard Definition", is a physical resolution in the 1280p*720p below a video format, refers to the vertical resolution of the video is 720 lines progressive scan. Specifically, refers to the resolution in the 400-line VCD, DVD, TV shows and other "SD" video format, that is, standard definition.
  • HD: While the physical resolution of more than 720p format is said to be high definition, the English Express definition, referred to as HD. As a "high-definition" entry-level standard, 720p refers to a video with a vertical resolution of 720 line-by-line scanning, which means that the TV screen must be able to display 720 horizontal lines simultaneously and completely. If expressed in the form of a common resolution of AXB in the display, the resolution of 720p must be at least 1280x720, and the usual resolutions of 1280x1024, 1280x768, 1280x800, and so on, are supported in 720p HD format. In turn, the resolution of the standard definition, we often touch the resolution of 640x480, 720x576 or 800x600, they are not in the vertical resolution of 720 lines, so it is not a "high definition" category. So, as long as the vertical resolution to reach 720 lines, can be called HD it? The answer is in the negative. On the standard of high definition, there are two internationally recognized: video vertical resolution of more than 720p or 1080i (p for progressive scan, I for interlaced scanning, this concept and CRT TV basically the same), while the video condone ratio of 16:9. For vertical resolution we have an intuitive understanding, and the so-called condone ratio, in fact, is the video screen "aspect ratio", if a rectangle to represent, then its length and width of the proportion must conform to 16:9--is commonly known as "widescreen" or "wide screen." By this ratio, the resolution of 720p is at least 1280x720, so that it meets the 16:9 requirement, so this is the standard resolution of 720p.
  • Look at the final summary: In fact, "HD" and "SD" can not be simple comparison, they are just two different video formats, they can be loaded with a variety of different quality video signal, so the actual quality and the quality of the video signal loaded has a great relationship, which is equivalent to water tanks and cups to fill, tank can be filled with a large tank It can also be a glass of water, when only a glass of water is not to say that it is loaded with water than a cup of water more.
• Video encoding
  
  • H. h.265 ...
  • H: Is MPEG-4 (MPEG-4 is a compression standard in MPEG format) Part I, a joint video group composed of the ITU-T video Coding Expert Group (VCEG) and ISO/IEC dynamic Image Expert Group (MPEG) (Jvt,joint video Team) proposed standard for highly compressed digital video codecs. This standard is often referred to as H.264/AVC (or avc/h.264 or h.264/mpeg-4 AVC or mpeg-4/h.264 AVC) and clearly describes its two developers.
  • The advantages of H:
    1. Low bit rate: compared with MPEG2 and MPEG4 ASP compression technology, in the same image quality, the amount of data compressed with H. 1/3 is only MPEG2 of 1/8,MPEG4.
    2. High-quality Images: H + + provides continuous, smooth, high-quality images (DVD quality).
    3. Strong fault tolerance: the necessary tools for solving errors such as packet loss that can occur in an unstable network environment are provided.
    4. Network adaptability: H. Abstraction provides a network abstraction layer, so that the files of H. E can be easily transmitted across different networks (e.g., Internet, cdma,gprs,wcdma,cdma2000, etc.).
    The biggest advantage is the high data compression ratio, in the same image quality conditions, the compression ratio of H. MPEG-2 is more than twice times, is the MPEG-4 1.5~2 times. For example, if the original file size of 88GB, using MPEG-2 compression standard compression into 3.5GB, compression ratio of 25:1, and the compression standard of H. 102:1 to 879MB, from 88GB to 879mb,h.264 compression ratio to achieve a staggering. The low bit rate, which plays an important role in the high compression ratio of H. MPEG-2 and MPEG-4 ASP compression technology, will greatly save users ' download time and data traffic charges. In particular, it is worth mentioning that the high compression ratio at the same time, but also has a high quality smooth image, because of this, after the compression of the video data, in the network transmission process requires less bandwidth and more economical.
  • H.265:H.265 is the new video coding standard developed by ITU-T Vceg following H. The h.265 standard revolves around the existing video coding standard, which retains some of the original technology while improving some of the relevant technologies. The new technology uses advanced techniques to improve the relationship between code flow, encoding quality, latency and algorithmic complexity to achieve optimal settings.
  • H.265 Features: The h.265 is designed to deliver higher quality network video at limited bandwidth, with only half the bandwidth available to play the same quality video. It also means that our smartphones, tablets and other mobile devices will be able to play 1080p Full HD video directly online. The h.265 standard also supports 4K (4096x2160) and 8K (8192x4320) Ultra HD video.
  • Attach a video coding development time chart:
  • 
    
• Audio encoding
  
  • AAC, MP3, AC3 ...
  • AAC: A file compression format designed for sound data, unlike MP3, which uses a new algorithm to encode, more efficiently, and with a higher "price/performance ratio". The use of AAC format, can make people feel the sound quality is not significantly reduced under the premise of more compact. Apple ipod, Nokia phone also support AAC format audio files.
    AAC Advantage: better sound quality compared to MP3,AAC format, smaller file size.
    AAC is not enough: AAC is lossy compression format, and the current popular ape, FLAC and other lossless format compared to the sound quality there is "essentially" gap. In addition, the faster transmission speed of USB3.0 and more than 16G large capacity MP3 is accelerating the popularity, but also makes the AAC "small" halo no longer exist.
  • Mp3:mp3 is an audio compression technology, the full name of the motion imaging expert compression standard audio Layer 3 (Moving Picture Experts Group Audio Layer III), referred to as MP3. It is designed to drastically reduce the amount of audio data. Using the technology of MPEG Audio Layer 3, the music is compressed into smaller files with 1:10 or even 1:12 compression, while for most users there is no significant decrease in the quality of playback compared to the original uncompressed audio.
  • MP3 Features: MP3 is the use of the ear to high-frequency sound signal is not sensitive to the characteristics of the time-domain waveform signal into the frequency domain signal, and divided into a plurality of frequency bands, different frequency bands using different compression ratio, high-frequency compression ratio (or even ignore the signal) to low-frequency signal using a small compression ratio, to ensure that the signal This is equivalent to abandoning the basic hearing of the ear of the high-frequency sound, [1] only to keep the low-frequency portion can be heard, so that the sound with 1:10 or even 1:12 compression rate compression. Since this compression method is called MPEG Audio Player3, it is referred to as MP3.
• Video Encapsulation Format
  
  • TS, m3u8 ...
    Often said TS Stream, or pass a m3u8 address come over, refers to the video packaging format. If placed in a browser request, it will be written as a stream file, you can open to see, inside is the broadcast of the video frame. You can also encapsulate the video as a TS file and then play it with the player. M3U8 can be multi-bitrate adaptation, according to network bandwidth, the client will choose a suitable for their own code rate of the file to play, to ensure the smooth flow of video.
• Multimedia playback components (Android)
  
  • MediaPlayer, Mediacodec, OMX, Stagefright
  • MediaPlayer: Playback control
  • Mediacodec: Audio and Video codec
  • OMX: The codec standard used in the multimedia section
  • Stagefright: A framework that replaces the previous opencore, mainly by making a OMX layer, just a reference to the omx-component part of Opencore. Stagefright is added to the Mediaplayerservice layer, and Opencore is tied together. Stagefright is present in the form of a shared library (libstagefright.so) in Android, where module–awesomeplayer can be used to play video/audio. Awesomeplayer provides many APIs that can be called by the upper-level application (JAVA/JNI). (This framework is described in detail later in this article)
• ... ...

Video section Related

• Resolution
  
  • The size of a frame of video, indicating the number of pixels (720x576, 1280x720, 1920x1080 ...)
  • The precision of the screen image refers to how many pixels a video image can display. Because the dots, lines, and polygons on the screen are made up of pixels, the more pixels you can display, the finer the picture, and the more information you'll see in the same screen area, so resolution is one of the most important performance indicators. The entire image can be imagined as a large chessboard, and the resolution is represented by the number of intersection points of all meridians and parallels. The display resolution is certain, the smaller the display, the clearer the image, conversely, when the display size is fixed, the higher the display resolution, the clearer the image.
• Frame rate
  
  • Number of video frames per second (24/25/30/48/60 FPS)
  • Because of the special physiological structure of human eyes, if the frame rate of the picture is higher than 24, it will be considered as coherent, and this phenomenon is called visual persistence. That's why the film is filmed in a single pane and then quickly played.
    And on the game, in general, the first person shooter game more attention to the height of FPS, if fps<30 words, the game will appear incoherent. So there is an interesting phrase: "fps (refers to FPS game) at FPS (referring to frame rate).
  • The number of frames per second (fps), or frame rate, indicates the number of times per second that the graphics processor is able to update when it processes the farm. The high frame rate allows for smoother, more realistic animations. Generally, 30fps is acceptable, but increasing the performance to 60fps can significantly improve the sense of interactivity and realism, but generally more than 75fps generally is not easy to detect a noticeable increase in fluency. If the frame rate exceeds the screen refresh rate, it will only waste graphics processing, because the monitor cannot be updated at such a fast rate, so the frame rate exceeding the refresh rate is wasted.
• Refresh Rate
  
  • Refresh rate: The speed at which the screen refreshes.
  • The lower the refresh rate, the more powerful the image flashes, pauses, and jitter, and the faster the eyes become fatigued.
    Use more than 70Hz refresh frequency to basically eliminate flicker, monitor best stable work at the highest allowable frequency, is generally 85Hz.
• Encoding format
  
  • Coding: The purpose is to compress data and compress redundant data by coding algorithm.
  • MPEG (MPEG-2, MPEG-4)
  • h.26x (H.263, H.264/AVC, H.265/HEVC)
• Package format
  
  • Encapsulates encoded audio and video data into a container in a certain format
  • Mkv/avi/ts ...
• Code rate
  
  • Number of bits per second (bps) of video data
  • File size (b) = bit rate (S/s) * duration
• Quality vs Bit Rate
  
  • The larger the bitrate, the better the picture, the smoother the video? Wrong
  • Video quality and bitrate, encoding algorithm are all related

This article is derived from the countercurrent fish yuiop:http://blog.csdn.net/hejjunlin/article/details/52431887

Audio section Related

• Encoding format
  
  • Lossy code: AAC/MP3/AMR/WMA
  • Lossless code: WAV/FLAC/APE/ALAC
  • Add: The data compressed with lossless encoding can be fully recovered, the decoded data is completely consistent with the original data, the compression is relatively small; lossy coding in the coding process to lose imperceptible information, and the lost data is not recoverable, compression ratio is large
• Quantization accuracy
  
  • The analog signal can be divided into a number of levels, the higher the accuracy of quantization, the music of the sound pressure amplitude closer to the original music.
  • The quantization accuracy of the BIT,CD standard is the quantization accuracy of the 16BIT,DVD standard is 24Bit.
  • It can also be understood as how many bits are represented by a sample point (8/16/24/32bit)
• Sample Rate
  
  • Number of audio sampling points per second (8000/44100hz)
  • The sampling rate unit is expressed in Hz (Hz)
• Number of channels
  
  • Stereo, 5.1,7.1 Channel
  • Stereo: Sound is assigned to two separate channels during recording, resulting in a good sound positioning effect. This technique is particularly useful in music appreciation, where listeners can clearly identify the direction in which the instruments come from, making the music more imaginative and closer to the spot. Stereo technology is widely used in a large number of sound cards since the acoustic Blaster Pro, becoming a far-reaching audio standard.
  • 5.1 Channel: In fact 5.1 sound system from 4.1 surround, the difference is that it adds a central unit. This central unit is responsible for transmitting less than 80Hz of sound signals, in the appreciation of the film is conducive to strengthening the voice, the dialogue in the middle of the entire sound field to increase the overall effect. Dolby Digital, DTS, and so on are all based on the 5.1 sound system as a technical blueprint for AC-3. Believe that every friend who truly experiences Dolby AC-3 sound will be impressed by the 5.1 channel. It is now widely used in traditional cinemas and home theaters.
• Audio Frames
  
  • A set of number of sample points
  • Coding: Basic Coding Unit
  • Example: The playback time of an audio frame = the number/sampling frequency (in s) of the sampled sample for an AAC frame.
    The sample rate (samplerate) is 44100Hz, representing 44,100 sample points per second,
    So, according to the formula,
    Playback duration of audio frames = number of sample points per AAC frame/Sample Frequency
    The current frame's playback time = 1024x768 * 1000000/44100= 22.32ms (in ms)
    48kHz Sample Rate:
    The current frame's playback time = 1024x768 * 1000000/48000= 21.32ms (in ms)
    22.05kHz Sample Rate:
    The current frame's playback time = 1024x768 * 1000000/22050= 46.43ms (in ms)

The first time to get blog update reminders, as well as more Android dry, source code Analysis , Welcome to follow my public number, sweep the bottom QR code or long press to identify two-dimensional code, you can pay attention to.

If you feel good, easy to praise, but also to the author's affirmation, can also share this public number to you more people, original not easy

Android Multimedia Framework Summary (v) Multimedia basic concept