FFmpeg Video Audio Codec 0 Basic Learning Method-B

Thank the great God to share, although now still do not understand, keep everyone together to see

PS: A lot of people don't know how to read "FFmpeg". It reads "EF EF em Peg"

0. Background knowledge

This chapter focuses on where ffmpeg are used (here are just a few of the many I know, in fact much more than this). To illustrate: FFmpeg is very important.

Use FFmpeg as the kernel video player:

Mplayer,ffplay, shooter player, Storm av, kmplayer,qq av ...

Use FFmpeg as the DirectShow Filter for the kernel:

Ffdshow,lav Filters ...

Use FFmpeg as the transcoding tool for the kernel:

FFmpeg, Format Factory ...

In fact, FFmpeg audio codec is really powerful, almost all of the existing audio and video coding standards, so as long as audio and video development, almost inseparable from it.

For those who do not have the background of the AV technology, it is best to understand some of the most basic audio-visual data formats before learning FFmpeg, and you can refer to the following article:

[Summary] AV codec technology 0 Basic Learning methods

Getting started with visual audio data processing: RGB, YUV pixel data processing

Getting Started with AV data processing: PCM Audio sampling data processing

Getting Started with AV data processing: Analysis of video stream in H.

Getting Started with AV data processing: AAC audio bitstream parsing

Getting Started with AV data processing: FLV Encapsulation Format parsing

Getting Started with AV data processing: UDP-RTP Protocol resolution

1. Use of FFmpeg programs (Ffmpeg.exe,ffplay.exe,ffprobe.exe)  

"Video Resources"

The 1th and 2 chapters of this paper are the basic content of ffmpeg programming. This part of the content I recorded as a substitute for the sophomore when the video, there is time to see the "video player based on the FFmpeg + SDL production" course video.

This chapter mainly introduces the use of the three EXE included in the FFmpeg project.

FFmpeg's official website is: http://ffmpeg.org/

Compiled version of Windows available (you can connect to this site, and the official website in sync): http://ffmpeg.zeranoe.com/builds/

The FFmpeg in this website is divided into 3 versions: Static,shared,dev.

The first two versions can be used directly on the command line, their difference is: Static inside only 3 applications: Ffmpeg.exe,ffplay.exe,ffprobe.exe, each EXE volume is very large, the relevant DLL has been compiled into the EXE. Shared in addition to 3 applications: Ffmpeg.exe,ffplay.exe,ffprobe.exe, there are some DLLs, such as Avcodec-54.dll. Shared EXE volume is very small, they are running, to the corresponding DLL to invoke the function.

The dev version is for development and contains the library file Xxx.lib as well as the header file Xxx.h, which does not contain EXE files.

Open the System command line interface, switch to the directory where the FFmpeg is located, you can use the 3 applications.

1.1 Ffmpeg.exe

FFmpeg is an application for transcoding.

A simple transcoding command can be written like this:

Transcode the Input.avi into Output.ts and set the video bitrate to 640kbps

1. Ffmpeg-i input.avi-b:v 640k Output.ts

Specific use of the method can be consulted: FFmpeg parameter Chinese detailed explanation

Detailed instructions for use (English): http://ffmpeg.org/ffmpeg.html

1.2 Ffplay.exe

Ffplay is the application for playback.

A simple play command can be written like this:

Play Test.avi

1. Ffplay Test.avi

The specific use of the method can be consulted: Ffplay shortcut keys and options

Detailed instructions for use (English): http://ffmpeg.org/ffplay.html

1.3 Ffprobe.exe

Ffprobe is an application that is used to view file formats.

This is not a lot of introduction.

Detailed instructions for use (English): http://ffmpeg.org/ffprobe.html

2. Use of the FFmpeg library: Video player

This chapter begins with the development of a library using FFmpeg.

2.1 Configuration of the FFmpeg library

From the http://ffmpeg.zeranoe.com/builds/website

1. Download the dev version, which contains the FFmpeg xxx.h header file and the Xxx.lib library file.

2. Download the shared version, which contains the ffmpeg DLL file.

3. Copy these two parts of the file to the VC project.

Note: Problems may occur, see: FFMPEG Library porting to VC required steps

If you don't want to configure yourself manually, you can download a project that's already configured: The simplest FFMPEG+SDL-based video player

2.2 The simplest video player

Learn the article "100 lines of code to achieve the simplest video player based on FFMPEG+SDL" code, this is ffmpeg do video player The simplest code, is my own streamlined, can no longer simplify, each line is very important.

The original is a SDL1.2-based video player, and later updated the simplest video player based on SDL2.0: The simplest ffmpeg+sdl-based video player Ver2 (with SDL2.0)

The above player uses Libavformat and Libavcodec two class library to complete the video decoding work. In fact, the decoding work only needs to be libavcodec. Therefore, a "pure" decoder is updated. The decoder works only with Libavcodec: The simplest ffmpeg-based decoder-pure version (not including Libavformat)

The function of FFmpeg: introduction of FFmpeg function

Note 1: SDL is used when playing video or audio data. For SDL, refer to: SDL Introduction

SDL Reference Document: SDL Guide Chinese translation

NOTE 2: If you want to view the decoded data, you need to use the YUV Player: YUV player source code or YUV player Deluxe can

A study of 2.3 related structural bodies

The relationship between FFmpeg structures reference article: Relationships between the most critical structures in FFmpeg

For the analysis of each variable in the structure, refer to the article:

FFMPEG Structure Analysis: Avframe

FFMPEG Structure Analysis: Avformatcontext
FFMPEG Structure Analysis: Avcodeccontext
FFMPEG Structure Analysis: Aviocontext
FFMPEG Structure Analysis: Avcodec
FFMPEG Structure Analysis: Avstream
FFMPEG Structure Analysis: Avpacket

3. Use of the FFmpeg library: Audio Player 3.1 The simplest audio player

Learn the code in the simplest FFMPEG+SDL-based audio player, like the simplest video player, which is the simplest audio player, and each line of code is important.

The original is a SDL1.2-based audio player and later updated a SDL2.0-based, simplest audio player: The simplest ffmpeg+sdl-based audio player Ver2 (with SDL2.0)

Note: If you want to view the decoded data (PCM data), you need to use the audition.

4. Use of the FFmpeg library: a real player--ffplay4.1 Real Players

Ffplay flowchart such as the article "Ffplay Source code Analysis: The overall flowchart" is shown. The Ffplay code is more complex, but its core code is the same as the 100-line code that implements the simplest ffmpeg+sdl-based video player. Two projects can be combined with learning.

Ffplay Code profile: How to write a simple player with ffmpeg

Ffplay Instructions for use: Ffplay Shortcuts and options

Ffplay has been transplanted to the VC under the project: ffplay_vc2005 (Others do, the quality is very good)

Ffplay porting to MFC works, including a simple graphical interface and some control buttons: Ffplay player porting VC Project: Ffplay for MFC

Code description of the above software: Ffplay for MFC code Memo

Simple analysis of FFPLAY.C function structure: simple analysis of FFPLAY.C function structure (drawing)

5. Use of the FFmpeg library: encoding5.1 Encoding

FFmpeg Coding I am not a lot of research, you can refer to the article: Using FFmpeg Class library to implement YUV video sequence encoding for video

The above article is used in the old class library, the use of the new version of the Library can refer to the following articles.

Image encoding can be referenced: The simplest ffmpeg-based image encoder (YUV encoded as JPEG)

Audio encoding can be referenced by: The simplest ffmpeg-based audio encoder (PCM encoded as AAC)

Video encoding can be referenced: The simplest video encoder based on FFmpeg (YUV encoded as H.

HEVC (h.265) video encoding can be referenced by: The simplest ffmpeg-based video encoder-Updated (YUV encoded as HEVC (h.265))

The encoder uses Libavformat and libavcodec two class libraries to complete the coding of the video. In fact, the coding work only needs to be libavcodec. Therefore, a "pure" encoder is updated. The encoder works only with Libavcodec: The simplest ffmpeg-based encoder-pure version (not including Libavformat)

5.2 transcoding

Transcoding is actually decoding and then coding first.

Programs that do not transcode and only package format conversions are available for reference: The simplest ffmpeg-based package Format converter (no codec)

Transcoding program can refer to: The simplest ffmpeg-based transcoding program

More complex transcoding program can refer to FFMPEG.C, it ported to MFC under the project: FFmpeg transcoding VC Project: FFmpeg for MFC

Simple analysis of FFMPEG.C function structure: simple analysis of FFMPEG.C function structure (drawing)

6. FFmpeg Source Code Analysis

After familiar with the use of FFmpeg library, you can look at the source code of FFmpeg. Note that the source code for FFmpeg can only be compiled under Linux, and it is compiled with MinGW under Windows. We recommend that you use Eclipse to view the source code for FFmpeg.

There is a very complete FFmpeg source code Analysis document: Ffdoc

The FFmpeg library function source code Analysis article lists the following:

"Architecture Diagram"

FFmpeg source code structure diagram-decoding

FFmpeg source code structure Map-Encoding

General

FFmpeg Source code Simple analysis: Av_register_all ()

FFmpeg Source code Simple analysis: Avcodec_register_all ()

FFmpeg Source code Simple analysis: allocation and release of memory (Av_malloc (), Av_free (), etc.)

Simple analysis of FFmpeg source code: Initialization and destruction of common structures (avformatcontext,avframe, etc.)

FFmpeg Source code Simple analysis: Avio_open2 ()

FFmpeg Source code Simple analysis: Av_find_decoder () and Av_find_encoder ()

FFmpeg Source code Simple analysis: Avcodec_open2 ()

FFmpeg Source code Simple analysis: Avcodec_close ()

Decoding

Diagram FFmpeg The function that opens the media avformat_open_input

FFmpeg Source code Simple analysis: Avformat_open_input ()

FFmpeg Source code Simple analysis: Avformat_find_stream_info ()

FFmpeg Source code Simple analysis: Av_read_frame ()

FFmpeg Source code Simple analysis: Avcodec_decode_video2 ()

FFmpeg Source code Simple analysis: Avformat_close_input ()

Encoding

FFmpeg Source code Simple analysis: Avformat_alloc_output_context2 ()

FFmpeg Source code Simple analysis: Avformat_write_header ()

FFmpeg Source code Simple analysis: Avcodec_encode_video ()

FFmpeg Source code Simple analysis: Av_write_frame ()

FFmpeg Source code Simple analysis: Av_write_trailer ()

Other

FFmpeg source code Simple analysis: Log output system (Av_log (), etc.)

FFmpeg source code Simple analysis: Structure member management System-avclass

FFmpeg source code Simple analysis: Structure member management System-avoption

FFmpeg source code Simple analysis: Libswscale's Sws_getcontext ()

FFmpeg source code Simple analysis: Libswscale's Sws_scale ()

FFmpeg source code Simple analysis: Libavdevice's Avdevice_register_all ()

FFmpeg source code Simple analysis: Libavdevice's Gdigrab

Script

FFmpeg source code Simple analysis: Makefile

FFmpeg source Code Simple analysis: Configure

A list of source code analysis articles for LIBAVCODEC below:

"Decode-libavcodec H. A decoder"

Simple analysis of the source code of the H. FFmpeg decoder: An overview

Simple analysis of the source code of the H. FFmpeg Decoder: Parser (Parser) section

Simple analysis of the source code of the H. FFmpeg decoder: The main part of the decoder

Simple analysis of the source code of the H. FFmpeg decoder: Entropy decoding (entropydecoding) part

Simple analysis of the source code of the H. FFmpeg Decoder: Macro block decoding (Decode) part-intra-frame macro block (Intra)

Simple analysis of the source code of the H. FFmpeg Decoder: Macro block Decoding (Decode) section-Inter-frame macro block (Inter)

Simple analysis of the source code of the H. FFmpeg Decoder: Loop filter (loopfilter) section

"Decode-libavcodec HEVC Decoder"

FFmpeg's HEVC decoder source code Simple analysis: Overview

FFmpeg's HEVC decoder source code Simple analysis: Parser (Parser) part

FFmpeg's HEVC decoder source code Simple analysis: Decoder backbone part

FFmpeg's HEVC decoder source code Simple analysis: CTU decoding (Ctudecode) part-pu

FFmpeg's HEVC decoder source code Simple analysis: CTU decoding (CTU Decode) part-tu

FFmpeg's HEVC decoder source code Simple analysis: Loop filter (Loopfilter)

7.FFmpeg use of several other classes of libraries 7.1.libavfilter (plus effects)

Avfilter can add various filter effects to visual audio. There are two examples, one is to add a watermark to the video:

Simplest FFmpeg-based Avfilter example (watermark overlay)

The other is to add special effects to YUV data:

Simplest example of a ffmpeg-based Avfilter-Pure edition

7.2.libavdevice (read device)

Avdevice can read data from a computer's multimedia device, or output data to a specified multimedia device.

Article using the Ffmpeg.exe command line tool directly: FFmpeg get DirectShow device data (camera, recording screen)

There are 2 examples of programming aspects:

Read the camera: The simplest ffmpeg-based Avdevice example (read camera)

Screen recording: The simplest ffmpeg-based Avdevice example (screen recording)

7.3.libswscale (image stretch, pixel format conversion)

The Swscale class Library can convert the format of pixel data while stretching the size of the image.

Example of use of Libswscale: The simplest example of FFmpeg-based Libswscale (YUV to RGB)

In addition, this example comes with a program to generate a test picture: The simplest example attachment for FFmpeg-based libswscale: Test picture Generation tool

Handling of 8.FFMPEG Package formats

The processing of encapsulation formats using FFmpeg is done primarily through Avformat. There are 3 examples of processing of encapsulation formats:

Encapsulated Format Converter: The simplest ffmpeg-based package Format converter (no codec)

AV Splitter Simplified version (demuxer-simple): Simplest ffmpeg-based package format processing: AV splitter simplified version (demuxer-simple)

AV Splitter (demuxer): The simplest ffmpeg-based package format processing: AV Splitter (demuxer)

AV multiplexer (muxer): The simplest ffmpeg-based package format processing: AV multiplexer (muxer)

9.FFmpeg Streaming Media applications

The use of ffmpeg for streaming media applications, mainly the transmission and reception of streaming media.

Articles directly using the Ffmpeg.exe command line tool:

ffmpeg command to send streaming media (udp,rtp,rtmp)

An example of programming is:

FFmpeg-based ejector: The simplest ffmpeg-based push-to-stream (for example, to push rtmp)

Other miscellaneous in 10.FFmpeg

Examples of using ffmpeg to read and write memory (not files):

Memory Player: The simplest example of memory read-write based on ffmpeg: Memory player

Memory transcoding: The simplest example of memory read-write based on ffmpeg: Memory transcoding

Application of FFmpeg in other platforms

The example programs that apply FFmpeg to Android, IOS, and Windows phone can be consulted:

The simplest mobile-based FFmpeg example: Android HelloWorld

The simplest mobile-based FFmpeg example: Android Video Decoder

The simplest mobile-based FFmpeg example: Android Video Decoder-Single library version

The simplest ffmpeg-based mobile example: Android push-to-stream

The simplest mobile-based FFmpeg example: Android video transcoding

The simplest ffmpeg-based mobile example attachment: Android comes with the player

The simplest ffmpeg mobile-based example attachment: SDL Android HelloWorld

The simplest mobile-based FFmpeg example: IOS HelloWorld

The simplest mobile-based FFmpeg example: IOS Video Decoder

The simplest mobile-based FFmpeg example: IOS streaming

The simplest mobile-based FFmpeg example: IOS video transcoding

The simplest ffmpeg-based mobile example attachment: iOS comes with a player

The simplest mobile-based FFmpeg example: Windows phonehelloworld

Study of FFMPEG related engineering

Learn to complete ffmpeg, you can also learn about ffmpeg based on the relevant multimedia open source project, here are recommended the following:

12.1 ffdshow

Ffdshow is a DirectShow Filter based on the FFmpeg decoder class library Libavcodec. Widely installed on the PC.

Source Code Analysis articles for ffdshow (in the update):

ffdshow Source Code Analysis 1: Overall structure
Ffdshow Source Code Analysis 2: Bitmap Overlay Filter (dialog Box Section Dialog)

Ffdshow Source Code Analysis 3: Bitmap overlay Filter (set part settings)
Ffdshow Source Code Analysis 4: Bitmap Overlay Filter (Filter section filter)
Ffdshow Source Code Analysis 5: Bitmap Overlay Filter (summary)
Ffdshow Source Code Analysis 6: Encapsulation of DLLs for decoders (LIBAVCODEC)
Ffdshow Source code Analysis 7:LIBAVCODEC Video Decoder class (TVIDEOCODECLIBAVCODEC)
Ffdshow Source Code Analysis 8: Video Decoder class (TVIDEOCODECDEC)
Ffdshow Source Code Analysis 9: Codec about class summary

12.2 LAV Filters

LAV Filter is a ffmpeg-based decoder class library Libavcodec, and a DirectShow filter for the Unpacking class library Libavformat. Widely installed on the PC.

Source Code Analysis Articles for LAV filter:

LAV Filter Source Code Analysis 1: Overall structure

LAV Filter Source code Analysis 2:lav Splitter

LAV Filter Source Code Analysis 3:lav Video (1)

LAV Filter Source Code Analysis 4:lav Video (2)

12.3 Mplayer

MPlayer is the most widely used player in Linux, as well as the Windows version. The ffmpeg is used.

Source code Analysis articles about MPlayer:

MPlayer Source Code Analysis

12.4 Media Player CLASSIC-HC

Many of the players that are now widely used are built on the basis of media player CLASSIC-HC.

Source code Analysis articles about media Player CLASSIC-HC:

Media Player CLASSIC-HC Source Code Analysis 1: Overall structure
Media Player CLASSIC-HC Source Code Analysis 2: Core Class (CMainFrame) (1)
Media Player CLASSIC-HC Source Code Analysis 3: Core Class (CMainFrame) (2)
Media Player CLASSIC-HC Source Code Analysis 4: Core Class (CMainFrame) (3)
Media Player CLASSIC-HC Source Code Analysis 5: About dialog boxes (CABOUTDLG)
Media Player CLASSIC-HC Source Code Analysis 6:mediainfo tab (CPPAGEFILEMEDIAINFO)
Media Player CLASSIC-HC Source Code Analysis 7: Details tab (cppagefileinfodetails)

12.5 XBMC

XBMC is an excellent free and open source (GPL) Media Center software.

For XBMC Source Code analysis articles:

XBMC Source Code Analysis 1: Overall structure and compilation method

XBMC Source Code Analysis 2:addons (skin skins)

XBMC Source Code Analysis 3: Core section-Overview

XBMC Source Code Analysis 4: Video player (Dvdplayer)-Decoder (take ffmpeg as an example)

XBMC Source Code Analysis 5: Video player (dvdplayer)-de-multiplexer (take ffmpeg as an example)

XBMC Source Code Analysis 6: Video player (dvdplayer)-File header (take FFmpeg as an example)

XBMC Source Code Analysis 7: Video player (dvdplayer)-input stream (take librtmp as an example)

FFmpeg AV codec 0 Basic Learning Method-B