FFmpeg Basic Usage

the problem to be solved in this lesson

What is the 1.FFmpeg transcoding process?
2. What do the common video formats contain?
3. How to extract the content from the video file.
4. How to convert from one format to another format.
5. How to zoom in and out of the video.
6. How to rotate, flip, fill, crop, blur, sharpen video.
7. How to add logo to the video, delete the logo.
8. How to add text to the video, dynamic text.
9. How to handle the picture.
10. How to record, add dynamic logo, screenshots, mosaic video. First Part

Basis
Terms
Container (Container)
A container is a file format, such as Flv,mkv. Contains the following 5 streams and file header information.
Flow (Stream)
is a video data transmission mode, 5 kinds of streams: audio, video, subtitles, attachments, data.
Frames (frame)
The frame represents a still image, divided into I-frames, p-frames, B-frames.
Codec (CODEC)
is to compress or decompress the video, CODEC =code (encoded) +decode (decoding)
For reuse/demultiplexer (MUX/DEMUX)
Put different streams in the container in accordance with the rules of a container, which is called multiplexing (MUX)
To parse a different stream from a container, this behavior is called a Demux introduction .

The name of FFmpeg comes from the MPEG video coding standard, the front "FF" stands for "Fast Forward", and FFmpeg is an open-source computer program that can be used to record, convert digital audio, video, and turn it into a stream. It is easy to convert multiple video formats to each other.

FFmpeg users have Google,facebook,youtube, Youku, iqiyi art, potatoes and so on.
Composition
1, Libavformat: For a variety of audio and video packaging format generation and resolution, including to obtain the information needed to decode the decoding context structure and read the audio and video frames and other functions, containing demuxers and Muxer library;
2, Libavcodec: For various types of sound/image codec;
3, Libavutil: Contains some common tool functions;
4, Libswscale: For video scene scaling, color mapping conversion;
5, Libpostproc: For the post-processing effect;
6, FFmpeg: is a command line tool for the video file conversion format, also supports real-time encoding of the TV card;
7, Ffsever: is an HTTP multimedia real-time broadcast streaming server, support time translation;
8, Ffplay: is a simple player, using the FFmpeg library parsing and decoding, through the SDL display;

FFmpeg processing Process
Filters (Filter)
In multimedia processing, filter means a software tool that is used to modify the contents of an input file before it is encoded into an output file. such as: Video flip, rotate, zoom and so on.
Syntax: [Input_link_label1][input_link_label2] ... filter_name=parameters [Output_link_label1][output_link_label2] ...
Filter Diagram Link Label: Is the name of the input or output of the tag filter
1. Video Filter-VF
If the TESTSRC video rotates 90 degrees in a clockwise direction
Ffplay-f lavfi-i TESTSRC-VF transpose=1
such as TESTSRC video horizontal flip (flip left and right)
Ffplay-f lavfi-i TESTSRC-VF Hflip
2. Audio Filter-AF
For slow playback, the sound speed is 50% of the original speed
Ffplay P629100.mp3-af atempo=0.5

How to achieve a clockwise rotation of 90 degrees and flip horizontally.
Filter Chain (Filterchain)
Basic syntax
Filterchain = A comma-delimited set of filter
Syntax: "Filter1,filter2,filter3,... Filtern-2,filtern-1,filtern"
Rotate 90 degrees clockwise and flip horizontally
Ffplay-f lavfi-i TESTSRC-VF Transpose=1,hflip

How to achieve horizontal flip video and source video for comparison. See how the filter chain is implemented.

Filter Chain (Filterchain)
The first step: The source video width is enlarged twice times.
Ffmpeg-i jidu.mp4-t 10-VF Pad=2*iw output.mp4
Step Two: Flip the source video horizontally
Ffmpeg-i jidu.mp4-t 10-VF Hflip output2.mp4
Step three: Horizontal Flip Video Overlay Output.mp4
Ffmpeg-i output.mp4-i Output2.mp4-filter_complex overlay=w compare.mp4
is not very complicated.
A filter diagram (filtergraph) with a link tag requires only one command.
Filter Diagram (filtergraph)
Basic syntax
Filtergraph = A semicolon-delimited set of Filterchain
"Filterchain1;filterchain2;...filterchainn-1;filterchainn"

Categories of Filtergraph
1. Simple and one-to-one
2, complex (complex) many-to-one, many-to-many
Filter Diagram (filtergraph)
Simple filter Diagram processing flow:

Complex filter diagram processing flow:

From the figure can be found that the complex filter diagram is less than a simple filter diagram 2 steps, efficiency than simple, ffmpeg recommended to use the complex filter diagram as far as possible.
Answer the above questions to achieve horizontal flip video and source video to compare

Filter Diagram (filtergraph)
View results directly with Ffplay:
Ffplay-f lavfi-i TESTSRC-VF Split[a][b]; [A]pad=2*iw[1]; [B]hflip[2]; [1] [2]overlay=w

F1:split filter creates a copy of two input files and marks them as [a],[b]
F2: [A] as input to the pad filter, the pad filter produces twice times the width and outputs to [1].
F3: [b] as input to the Hflip filter, the Vflip filter flips the video horizontally and outputs to [2].
F4: Use the overlay filter to cover [2] next to [1].

Select Media Stream
Some multimedia containers, such as avi,mkv,mp4, can contain multiple streams of different kinds, and how to extract various streams from the container.
Grammar:
-map File_number:stream_type[:stream_number]

Here are some descriptions of the special flow symbols:
1,-map 0 Select all the streams for the first file
2,-map i:v get all the video stream from the file ordinal I (index),-map i:a get all the audio stream,-map i:s get all the caption stream and so on.
3, the special parameter-an,-vn,-sn excludes all audio, video, subtitle stream respectively.

Note: The file sequence number and the stream ordinal count from 0 onwards.
Select Media Stream Part two

View Help
Help
The FFmpeg tool has a huge console help. The following table describes some of the options that are available, in italics, for the items to be replaced, and Ffplay and ffprobe have similar options.

Help
Available bit streams: FFMPEG–BSFS
Codecs available: Ffmpeg–codecs
Available decoders: Ffmpeg–decoders
Available encoders: ffmpeg–encoders
Available Filters: Ffmpeg–filters
Available Video formats: Ffmpeg–formats
Available channel layouts: ffmpeg–layouts
Available License:ffmpeg–l
Available pixel formats: FFMPEG–PIX_FMTS
Available protocols: Ffmpeg-protocals Part III

Bitrate, frame rate, and file size
Overview
Bitrate and frame rate are the most important basic features of video files, and their unique settings determine video quality. If we know the bitrate and the length of time then it is easy to calculate the size of the output file.

Frame rate: Frame rate is also called frame frequency, frame rate is the number of frames per second in the video file, the naked eye want to see a continuous moving image needs at least 15 frames.
Bitrate: Bit rate (also called code rate, data rate) is a parameter that determines the overall video/audio quality, the number of bytes processed in seconds, the bitrate is proportional to the video quality, and the bit rate in the video file is expressed in bps.
Frame rate
1. Set the frame rate with the-R parameter
Ffmpeg–i input–r fps output
2. Set frame rate with FPS filter
Ffmpeg-i CLIP.MPG-VF fps=fps=25 CLIP.WEBM
Predefined values for frame rate:

For example, the set code rate is 29.97fps, the following three ways have the same result:
Ffmpeg-i input.avi-r 29.97 Output.mpg
Ffmpeg-i input.avi-r 30000/1001 Output.mpg
Ffmpeg-i input.avi-r NETSC Output.mpg

Code rate, File size
Setting the bitrate –b parameter
-B
Ffmpeg-i film.avi-b 1.5M Film.mp4

Audio:-b:a Video:-B:v
Set video bitrate to 1500kbps
Ffmpeg-i input.avi-b:v 1500k Output.mp4

Control output File Size
-FS (file size initials)
Ffmpeg-i Input.avi-fs 1024K Output.mp4
Calculate output File Size
(Video bitrate + audio bitrate) * Duration/8 = file size K part fourth

Adjust video resolution
Adjust video resolution
1, set the video resolution with the-s parameter, the parameter value wxh,w width unit is pixel, h height unit is pixel
Ffmpeg-i input_file-s output_file

2, pre-defined video size
The following two commands have the same effect
Ffmpeg-i input.avi-s 640x480 Output.avi
Ffmpeg-i input.avi-s VGA Output.avi

The following table lists all the predefined dimensions

Scale Filter Adjust resolution
The advantage of scale filter is that you can use some additional parameters
Grammar:
SCALE=WIDTH:HEIGHT[:INTERL={1|-1}]
The following table lists the common additional parameters
Example
The following two commands have the same effect
Ffmpeg-i input.mpg-s Output.mp4
Ffmpeg-i INPUT.MPG-VF scale=320:240 Output.mp4

Scale the input video proportionally
Change to source video half size
Ffmpeg-i INPUT.MPG-VF SCALE=IW/2:IH/2 Output.mp4
Change the size of the original video to 90%:
Ffmpeg-i INPUT.MPG-VF scale=iw*0.9:ih*0.9 Output.mp4

Example
At the resolution of unknown video, the resolution of the adjustment is guaranteed to have the same cross-aspect ratio as the source video.
Width fixed 400, height proportional:
Ffmpeg-i INPUT.AVI-VF scale=400:400/a
Ffmpeg-i INPUT.AVI-VF scale=400:-1

In contrast, the height is fixed at 300 and the width is proportional:
Ffmpeg-i INPUT.AVI-VF scale=-1:300
Ffmpeg-i INPUT.AVI-VF scale=300*a:300 part fifth

Crop/Fill Video
Cropping Video crop filter
Select the rectangular area you want from the input file to the output file, commonly used to go to the black edge of the video.
Grammar: Crop:ow[:oh[:x[:y:[:keep_aspect]]

Example
Clip the input video to the left One-third, Middle One-third, and one of the right three points:
Ffmpeg-i INPUT-VF crop=iw/3:ih:0:0 Output
Ffmpeg-i INPUT-VF crop=iw/3:ih:iw/3:0 Output
Ffmpeg-i INPUT-VF crop=iw/3:ih:iw/3*2:0 Output
Crop the center of a frame
When we want to crop the area in the middle of the frame, the crop filter can skip the input x and Y values, and their default values are
Xdefault = (input width-output width)/2
Ydefault = (input height-output height)/2
Ffmpeg-i input_file-v crop=w:h output_file
Crop half the middle area:
Ffmpeg-i INPUT.AVI-VF CROP=IW/2:IH/2 Output.avi

Example
Compare cropped video and source video comparison
Ffplay-i JIDU.MP4-VF Split[a][b]; [A]drawbox=x= (IW-300)/2: (ih-300)/2:w=300:h=300:c=yellow[a]; [A]pad=2*iw[c]; [b]crop=300:300: (iw-300)/2: (ih-300)/2[b]; C [b]overlay=w*2.4:40

Automatically detect cropping areas
Cropdetect Filter Auto Detect black Edge area
Ffplay JIDU.MP4-VF Cropdetect

Then use the detected values to crop the video
Ffplay JIDU.MP4–VF crop=672:272:0:54

Fill Video (PAD)
Adds a quick extra area to the video frame, often showing different aspect ratios when playing
Grammar: Pad=width[:height:[:x[:y:[:color]]

Example
Create a 30-pixel pink width to surround a SVGA-sized picture:
Ffmpeg-i PHOTO.JPG-VF Pad=860:660:30:30:pink framed_photo.jpg
?

Similarly, you can make testsrc video with 30 pixels of pink surround video
Ffplay-f lavfi-i TESTSRC-VF Pad=iw+60:ih+60:30:30:pink

4:3 to 16:9
Some devices can only play 16:9 of the cross-aspect ratio, 4:3 of the cross-aspect ratio must be in the horizontal direction of the two sides filled with 16:9,

Height is maintained, width equals height multiplied by 16/9,x (input file horizontal displacement) The value is computed by the expression (output_width-input_width)/secondly.

The general commands from 4:3 to 16:9 are:
Ffmpeg-i INPUT-VF Pad=ih*16/9:ih:(ow-iw)/2:0:color output

Example
Ffplay-f lavfi-i testsrc-vf pad=ih*16/9:ih: (OW-IW)/2:0:pink

16:9 to 4:3
In order to display a 16:9 aspect ratio with a 4:3 aspect ratio, fill the vertical sides of the input file, the width remains constant, and the height is the width of the 3/4,y value (the vertical offset of the input file) is calculated by an expression (Output_height-input_height)/2.

Common commands from 16:9 to 4:3:
Ffmpeg-i input-vf pad=iw:iw*3/4:0: (oh-ih)/2:color output

Example
Ffplay-f lavfi-i testsrc=size=320x180-vf pad=iw:iw*3/4:0: (oh-ih)/2:pink part sixth

Flip and rotate
Flip
Horizontal Flip Syntax:-VF hflip
Ffplay-f lavfi-i TESTSRC-VF Hflip

Vertical Flip Syntax:-VF vflip
Ffplay-f lavfi-i TESTSRC-VF Vflip

Rotating
Syntax: transpose={0,1,2,3}
0: Rotate 90° counterclockwise and flip vertically
1: Clockwise rotation 90°
2: Rotate counterclockwise 90°
3: Rotate 90° Clockwise and flip the seventh part horizontally

Blur, sharpen
Fuzzy
Syntax: Boxblur=luma_r:luma_p[:chroma_r:chram_p[:alpha_r:alpha_p]]
Ffplay-f lavfi-i TESTSRC-VF Boxblur=1:10:4:10
Note: The range of Luma_r and alpha_r radius is 0~min (w,h)/2, Chroma_r radius is 0~min (cw/ch)/2
Sharpening
Syntax:-VF unsharp=l_msize_x:l_msize_y:l_amount:c_msize_x:c_msize_y:c_amount
All parameters are optional and the default value is 5:5:1.0:5:5:0.0
L_msize_x: Horizontal luminance matrix, value range 3-13, default value is 5
L_msize_y: Vertical luminance matrix, value range 3-13, default value is 5
L_amount: Brightness intensity, value range -2.0-5.0, negative number is blur effect, default value 1.0
c_msize_x: Horizontal color matrix, value range 3-13, default value 5
C_msize_y: Vertical color matrix, value range 3-13, default value 5
C_amount: Color intensity, value range -2.0-5.0, negative number is blur effect, default value 0.0
Example
Using default values, the luminance matrix is 5x5 and the luminance value is 1.0
Ffmpeg-i INPUT-VF Unsharp Output.mp4
Gaussian blur effect (stronger than the blur):
Ffplay-f lavfi-i TESTSRC-VF unsharp=13:13:-2 part eighth

Overlay (PIP)
Covered
Syntax: Overlay[=x[:y]
All parameters are optional and the default value is 0

Example
Logo in the top left corner
Ffmpeg-i pair.mp4-i Logo.png-filter_complex Overlay Pair1.mp4

Example
upper right Corner:
Ffmpeg-i pair.mp4-i Logo.png-filter_complex overlay=w-w pair2.mp4
Lower left corner:
Ffmpeg-i pair.mp4-i Logo.png-filter_complex overlay=0:h-h pair2.mp4
Lower right corner:
Ffmpeg-i pair.mp4-i Logo.png-filter_complex overlay=w-w:h-h pair2.mp4

Remove logo
Syntax:-VF Delogo=x:y:w:h[:t[:show]]
X:y the coordinates from the upper left corner
W:h logo width and height
T: Thickness of Rectangle Edge default value 4
Show: If set to 1 has a green rectangle, the default value is 0.
Ffplay-i JIDU.MP4-VF delogo=50:51:60:60:100:0 part nineth

Add text
Add text
Syntax:
drawtext=fontfile=font_f:text=text1[:p 3=v3[:p 4=v4[...]
Common parameter values
x: Horizontal axis from upper left corner
Y: Ordinate from upper left corner
FontColor: Font color
fontsize: Font size
Text: text content
Textfile: Text File
T: Timestamp, per second
N: Number of frames starting at 0
Draw/enable: Control file display, if the value is 0 not displayed, 1 is shown, you can use the function
Simple usage
1, add welcome text in the upper left corner
Ffplay-f LAVF I-i COLOR=C=WHITE-VF drawtext=fontfile=arial.ttf:text=welcom
2, add Good day
Ffplay-f lavfi-i color=c=white in the center -VF drawtext= "fontfile=arial.ttf:text= ' GoodDay ': x= (W-TW)/2:y= (h-th)/2"
3, setting font color and size
ffplay-f lavfi-i color=c =WHITE-VF drawtext= "fontfile=arial.ttf:text= ' Happy Holidays ': x= (W-TW)/2:y= (h-th)/2:fontcolor=green:fontsize=30"

Dynamic text
Implementing dynamic text with T (time-second) variables
1. Top horizontal scrolling
Ffplay-i jidu.mp4-vf drawtext= "fontfile=arial.ttf:text= ' Dynamic RTL text ': x=w-t*50:fontcolor=darkorange:fontsize= 30 "
2. Horizontal scrolling at the bottom
Ffplay-i jidu.mp4-vf drawtext= "Fontfile=arial.ttf:textfile=textfile.txt:x=w-t*50:y=h-th:fontcolor=darkorange: Fontsize=30 "
3. Scroll vertically from bottom to top
Ffplay jidu.mp4-vf drawtext= "textfile=textfile:fontfile=arial.ttf:x= (W-TW)/2:y=h-t*100:fontcolor=white:fontsize= 30 "
You want to implement the current time in the upper right corner.

Dynamic text
Display the current time in the upper-right corner localtime
Ffplay jidu.mp4-vf drawtext= "fontfile=arial.ttf:x=w-tw:fontcolor=white:fontsize=30:text= '%{localtime:%H\:%M\:%S} ’“

Displays the current time every 3 seconds
Ffplay jidu.mp4-vf drawtext= "fontfile=arial.ttf:x=w-tw:fontcolor=white:fontsize=30:text= '%{localtime:%H\:%M\:%S} ': Enable=lt (mod (t,3), 1) " part tenth

Image processing
Image support
FFmpeg supports most image processing, except for ljpeg (lossless JPEG), which can be decoded except exr,pic,ptx, all of which can be encoded.
Capture a picture using the –SS (seek from start) parameter.
Ffmpeg-ss 01:23:45-i Jidu.mp4 image.jpg
Create a GIF image from your video
Ffmpeg-i jidu.mp4-t 10-pix_fmt rgb24 jidu.gif
Convert video to picture (one picture per frame)
Ffmpeg-i Clip.avi frame%4d.jpg
Convert Pictures to Videos
Ffmpeg-f image2-i img%4d.jpg-r Video.mp4

Crop, fill
As with videos, images can be cropped and filled
Cutting
Ffmpeg-f lavfi-i RGBTESTSRC-VF crop=150:150 crop_rg.png
Fill
Ffmpeg-f lavfi-i SMPTEBARS-VF Pad=360:280:20:20:orange pad_smpte.jpg

Flip, rotate, overwrite
Same as the video picture can flip, rotate and overlay
Flip
Ffmpeg-i ORANGE.JPG-VF Hflip orange_hfilp.jpg
Ffmpeg-i ORANGE.JPG-VF Vflip orange_vfilp.jpg
Rotating
Ffmpeg-i IMAGE.PNG-VF transpose=1 Image_rotated.png
Covered
Ffmpeg-f lavfi-i rgbtestsrc-s 400x300 RGB. png
Ffmpeg-f lavfi-i Smptebars Smpte.png
Ffmpeg-i RGB. Png-i smpte.png-filter_complex overlay= (w-w)/2: (h-h)/2 rgb_smpte.png part 11th

Other advanced Techniques
Screen recording
Display device Name
Ffmpeg-list_devices 1-f dshow-i Dummy
Call camera
Ffplay-f dshow-i video= "Integrated Camera"
Save As File
Ffmpeg-y-F dshow-s 320x240-r 25-i video= "Integrated Camera"-b:v 800k-vcodec MPEG4 new.mp4

Add caption Subtitles
Grammar –VF Subtitles=file

Ffmpeg-i JIDU.MP4-VF subtitles=rgb.srt Output.mp4
Video tremor, color balance
Video tremor
Ffplay–i jidu.mp4-vf CROP=IN_W/2:IN_H/2: (in_w-out_w)/2+ ((In_w-out_w)/2) *sin (N/10):(In_h-out_h)/2 + ((in_h-out_h)/2 ) *sin (N/7)

Color balance
Ffplay-i JIDU.MP4-VF Curves=vintage
changing colors
Ffplay-i jidu.mp4-vf hue= "H=2*pi*t:s=sin (2*pi*t) +1"
Color conversion Black and white
Ffplay-i jidu.mp4-vf lutyuv= "u=128:v=128"
Set Audio video playback speed
3 times times video playback video
Ffplay-i JIDU.MP4-VF SETPTS=PTS/3
? Speed Play Video
Ffplay-i jidu.mp4-vf setpts=pts/(3/4)
Twice times the speed of playing audio
Ffplay-i Speech.mp3-af atempo=2

Problem: Video and audio simultaneously 3/4 slow playback
Screenshots
Cut a picture every second
Ffmpeg-i input.flv-f IMAGE2-VF Fps=fps=1 out%d.png
Cut a picture every 20 seconds
Ffmpeg-i input.flv-f IMAGE2-VF fps=fps=1/20 out%d.png

Note: The FFmpeg version n-57961-gec8e68c can be truncated at a maximum of 20s per image.
Multiple screenshots merged into a single file (2x3)? Every 1000 frames (seconds =1000/fps25) is a 40s truncation of a graph
Ffmpeg? -I jidu.mp4-frames 3-vf "Select=not (mod (n,1000)), scale=320:240,tile=2x3" Out.png

Mosaic Video
Mosaic Video
Create a mosaic video with multiple input files:
Ffmpeg-i jidu.mp4-i jidu.flv-i "Day by day Sbs.mp4"-i "Dangerous.mp4"-filter_complex "nullsrc=size=640x480 [base]; [0:v] setpts=pts-startpts, scale=320x240 [Upperleft]; [1:v] setpts=pts-startpts, scale=320x240 [upperright]; [2:v] setpts=pts-startpts, scale=320x240 [Lowerleft]; [3:v] setpts=pts-startpts, scale=320x240 [lowerright]; [Base] [Upperleft] overlay=shortest=1 [TMP1]; [TMP1] [Upperright] overlay=shortest=1:x=320 [TMP2]; [TMP2] [Lowerleft] overlay=shortest=1:y=240 [Tmp3]; [Tmp3] [Lowerright] overlay=shortest=1:x=320:y=240 "-c:v libx264 output.mkv

Dynamic logo Movement
The logo moves from left to right after 1 or 2 seconds:
Ffplay-i JIDU.MP4-VF Movie=logo.png[logo]; [In] [logo]overlay=x= ' If (GTE (t,2), ((t-2) *80)-w,nan) ': y=0

2, 2 seconds after the logo moved from left to right after the stop in the upper left corner
Ffplay-i JIDU.MP4-VF Movie=logo.png[logo]; [In] [logo]overlay=x= ' If (GTE ((((t-2) *80)-w,w), 0, ((t-2) *80)-W) ': y=0
3, every 10 seconds alternately appear logo.
Ffmpeg-y-T 60-i jidu.mp4-i logo.png-i Logo2.png-filter_complex "overlay=x=if (LT (mod (t,20), ten), 10,nan): y=10,overlay= X=if (GT (mod (t,20), W-w-10,nan): y=10 "Overlay.mp4

Information
FFmpeg official website: http://www.ffmpeg.org
FFmpeg doc:http://www.ffmpeg.org/documentation.html
FFmpeg Wiki:https://trac.ffmpeg.org/wiki
FAQ
Thanks.