Simple Analysis of FFmpeg source code: gdigrab of libavdevice,

Simple Analysis of FFmpeg source code: gdigrab of libavdevice,

This document records the source code of the GDIGrab component in libavdevice of FFmpeg. GDIGrab is used for screen recording (capturing) in Windows ). For use in ffmpeg.exe, refer to the article:

Use FFmpeg to obtain data from the DirectShow device (camera, screen recording)

For programming, refer to the following articles:

The simplest FFmpeg-based AVDevice example (screen recording)

The source code of gdigrab is located in libavdevice \ gdigrab. c. Shows the call relationship diagram of key functions. In the figure, the green background function represents the self-declared function in the source code, and the purple background function represents the Win32 API function.

Ff_gdigrab_demuxer in FFmpeg, the Device is also considered as a Format, because GDIGrab is an input Device and therefore is considered as an AVInputFormat. The AVInputFormat struct corresponding to GDIGrab is as follows.

AVInputFormat ff_gdigrab_demuxer = {    .name           = "gdigrab",    .long_name      = NULL_IF_CONFIG_SMALL("GDI API Windows frame grabber"),    .priv_data_size = sizeof(struct gdigrab),    .read_header    = gdigrab_read_header,    .read_packet    = gdigrab_read_packet,    .read_close     = gdigrab_read_close,    .flags          = AVFMT_NOFILE,    .priv_class     = &gdigrab_class,};

From this struct, we can see that:

The device name is "gdigrab ";
The complete device name is "gdi api Windows frame grabber ";
The initialization function pointer read_header () points to gdigrab_read_header ();
The Data Reading function pointer read_packet () points to gdigrab_read_packet ();
Close the function pointer read_close () pointing to gdigrab_read_close ();
Set Flags to AVFMT_NOFILE;
AVClass is specified as gdigrab_class.

Analyze the data below.

Gdigrab_classff_gdigrab_demuxer specifies its AVClass as a static variable named "gdigrab_class. The AVClass concept has been recorded before and will not be repeated here. Gdigrab_class is defined as follows.

static const AVClass gdigrab_class = {    .class_name = "GDIgrab indev",    .item_name  = av_default_item_name,    .option     = options,    .version    = LIBAVUTIL_VERSION_INT,};

According to the definition of gdigrab_class, it specifies an array named "options" as its option array (assigned to the option variable of AVClass ).

The following describes the definition of the options array.

#define OFFSET(x) offsetof(struct gdigrab, x)#define DEC AV_OPT_FLAG_DECODING_PARAMstatic const AVOption options[] = {    { "draw_mouse", "draw the mouse pointer", OFFSET(draw_mouse), AV_OPT_TYPE_INT, {.i64 = 1}, 0, 1, DEC },    { "show_region", "draw border around capture area", OFFSET(show_region), AV_OPT_TYPE_INT, {.i64 = 0}, 0, 1, DEC },    { "framerate", "set video frame rate", OFFSET(framerate), AV_OPT_TYPE_VIDEO_RATE, {.str = "ntsc"}, 0, 0, DEC },    { "video_size", "set video frame size", OFFSET(width), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, DEC },    { "offset_x", "capture area x offset", OFFSET(offset_x), AV_OPT_TYPE_INT, {.i64 = 0}, INT_MIN, INT_MAX, DEC },    { "offset_y", "capture area y offset", OFFSET(offset_y), AV_OPT_TYPE_INT, {.i64 = 0}, INT_MIN, INT_MAX, DEC },    { NULL },};

The options array contains the options supported by the Device. GDIGrab supports the following options:

Draw_mouse: Draw the mouse pointer.
Show_region: draw the boundary of the screen capture area.
Framerate: Specifies the frame capture rate.
Video_size: the size of the captured screen.
Offset_x: X-axis coordinate of the starting point of the screen capture.
Offset_y: Y coordinate of the starting point of the screen capture.

We can see from the macro definition "# define OFFSET (x) offsetof (struct gdigrab, x)" that these options are stored in a struct named "gdigrab.

Gdigrab Context Structure stores various variables used by the GDIGrab device. The definitions are as follows.

/** * GDI Device Demuxer context */struct gdigrab {    const AVClass *class;   /**< Class for private options */    int        frame_size;  /**< Size in bytes of the frame pixel data */    int        header_size; /**< Size in bytes of the DIB header */    AVRational time_base;   /**< Time base */    int64_t    time_frame;  /**< Current time */    int        draw_mouse;  /**< Draw mouse cursor (private option) */    int        show_region; /**< Draw border (private option) */    AVRational framerate;   /**< Capture framerate (private option) */    int        width;       /**< Width of the grab frame (private option) */    int        height;      /**< Height of the grab frame (private option) */    int        offset_x;    /**< Capture x offset (private option) */    int        offset_y;    /**< Capture y offset (private option) */    HWND       hwnd;        /**< Handle of the window for the grab */    HDC        source_hdc;  /**< Source device context */    HDC        dest_hdc;    /**< Destination, source-compatible DC */    BITMAPINFO bmi;         /**< Information describing DIB format */    HBITMAP    hbmp;        /**< Information on the bitmap captured */    void      *buffer;      /**< The buffer containing the bitmap image data */    RECT       clip_rect;   /**< The subarea of the screen or window to clip */    HWND       region_hwnd; /**< Handle of the region border window */    int cursor_error_printed;};

Gdigrab_read_header () is used to initialize gdigrab. The function is defined as follows.

/*** Initializes the gdi grab device demuxer (public device demuxer API ). ** @ param s1 Context from avformat core * @ return AVERROR_IO error, 0 success */static intgdigrab_read_header (AVFormatContext * s1) {struct gdigrab * gdigrab = s1-> priv_data; // window handle HWND hwnd; HDC source_hdc = NULL; HDC dest_hdc = NULL; BITMAPINFO bmi; HBITMAP hbmp = NULL; void * buffer = NULL; const char * filename = s1-> filen Ame; const char * name = NULL; AVStream * st = NULL; int bpp; RECT virtual_rect; // RECT clip_rect; BITMAP bmp; int ret; // filename is the window name if (! Strncmp (filename, "title =", 6) {name = filename + 6; // find the window handle hwnd = FindWindow (NULL, name); if (! Hwnd) {av_log (s1, AV_LOG_ERROR, "Can't find window '% s', aborting. \ n ", name); ret = AVERROR (EIO); goto error;} if (gdigrab-> show_region) {av_log (s1, AV_LOG_WARNING, "Can't show region when grabbing a window. \ n "); gdigrab-> show_region = 0;} // the filename is desktop} else if (! Strcmp (filename, "desktop") {// The Window handle is NULL hwnd = NULL;} else {av_log (s1, AV_LOG_ERROR, "Please use \" desktop \ "or \" title = <windowname> \ "to specify your target. \ n "); ret = AVERROR (EIO); goto error;} if (hwnd) {GetClientRect (hwnd, & virtual_rect);} else {// The Window handle is NULL, full Screen virtual_rect.left = GetSystemMetrics (SM_XVIRTUALSCREEN); virtual_rect.top = GetSystemMetrics (SM_YVIRTUALSCREEN); virtual_re Ct. right = virtual_rect.left + GetSystemMetrics (SM_CXVIRTUALSCREEN); response = virtual_rect.top + GetSystemMetrics (SM_CYVIRTUALSCREEN);}/* If no width or height set, use full screen/window area */if (! Gdigrab-> width |! Gdigrab-> height) {region = virtual_rect.left; region = virtual_rect.top; region = virtual_rect.right; region = region;} else {region = gdigrab-> offset_x; region = gdigrab-> offset_y; clip_rect.right = gdigrab-> width + gdigrab-> offset_x; height = gdigrab-> height + gdigrab-> offset_y;} if (clip_rect.left <virtual_rect.left | Constraint <virtual_rect.top | clip_rect.right> virtual_rect.right | response> virtual_rect.bottom) {av_log (s1, AV_LOG_ERROR, "Capture area (% li, % li), (% li, % li) extends outside window area (% li, % li), (% li, % li) ", region, region, clip_rect.right, region, virtual_rect.left, virtual_rect.top, virtual_rect.right, virtual_rect.bottom ); ret = AVERROR (EIO); goto err Or;}/* This will get the device context for the selected window, or if * none, the primary screen * // obtain the DC source_hdc = GetDC (hwnd); if (! Source_hdc) {WIN32_API_ERROR ("Couldn't get window device context"); ret = AVERROR (EIO); goto error;} bpp = GetDeviceCaps (source_hdc, BITSPIXEL); if (name) {av_log (s1, AV_LOG_INFO, "Found window % s, capturing % lix % I at (% li, % li) \ n", name, clip_rect.right-clip_rect.left, clip_rect.bottom-clip_rect.top, bpp, clip_rect.left, clip_rect.top);} else {av_log (s1, AV_LOG_INFO, "Capturing whole d Esktop as % lix % I at (% li, % li) \ n ", clip_rect.right-clip_rect.left, response-encoding, bpp, clip_rect.left, clip_rect.top );} if (clip_rect.right-clip_rect.left <= 0 | response-clip_rect.top <= 0 | bpp % 8) {av_log (s1, AV_LOG_ERROR, "Invalid properties, aborting \ n "); ret = AVERROR (EIO); goto error;} // create a HDC dest_hdc = CreateCompatibleDC (source_hdc) compatible with the specified device; if (! Dest_hdc) {WIN32_API_ERROR ("Screen DC CreateCompatibleDC"); ret = AVERROR (EIO); goto error ;} /* Create a DIB and select it into the dest_hdc * // BMP bmi. bmiHeader. biSize = sizeof (BITMAPINFOHEADER); bmi. bmiHeader. biWidth = clip_rect.right-clip_rect.left; bmi. bmiHeader. biHeight =-(clip_rect.bottom-clip_rect.top); bmi. bmiHeader. biPlanes = 1; bmi. bmiHeader. biBitCount = bpp; bmi. bmiHeader. biComp Ression = BI_RGB; bmi. bmiHeader. biSizeImage = 0; bmi. bmiHeader. biXPelsPerMeter = 0; bmi. bmiHeader. biYPelsPerMeter = 0; bmi. bmiHeader. biClrUsed = 0; bmi. bmiHeader. biClrImportant = 0; hbmp = CreateDIBSection (dest_hdc, & bmi, DIB_RGB_COLORS, & buffer, NULL, 0); if (! Hbmp) {WIN32_API_ERROR ("Creating DIB Section"); ret = AVERROR (EIO); goto error;} if (! SelectObject (dest_hdc, hbmp) {WIN32_API_ERROR ("SelectObject"); ret = AVERROR (EIO); goto error;}/* Get info from the bitmap */GetObject (hbmp, sizeof (BITMAP), & bmp); // create AVStream st = avformat_new_stream (s1, NULL); if (! St) {ret = AVERROR (ENOMEM); goto error;} avpriv_set_pts_info (st, 64, 1, 1000000 ); /* 64 bits pts in us * // Save the information to GDIGrab Context Structure gdigrab-> frame_size = bmp. bmWidthBytes * bmp. bmHeight * bmp. bmPlanes; gdigrab-> header_size = sizeof (BITMAPFILEHEADER) + sizeof (BITMAPINFOHEADER) + (bpp <= 8? (1 <bpp): 0) * sizeof (RGBQUAD)/* palette size */; gdigrab-> time_base = av_inv_q (gdigrab-> framerate ); gdigrab-> time_frame = av_gettime ()/av_q2d (gdigrab-> time_base); gdigrab-> hwnd = hwnd; gdigrab-> expires = source_hdc; gdigrab-> expires = expires; gdigrab-> hbmp = hbmp; gdigrab-> bmi = bmi; gdigrab-> buffer = buffer; gdigrab-> clip_rect = clip_rect; gdigrab-> cursor_error_printed = 0; if (gdigrab-> show_region) {if (then (s1, gdigrab) {ret = AVERROR (EIO); goto error ;}} st-> codec-> codec_type = AVMEDIA_TYPE_VIDEO; st-> codec-> codec_id = AV_CODEC_ID_BMP; st-> codec-> time_base = gdigrab-> time_base; st-> codec-> bit_rate = (gdigrab-> header_size + gdigrab-> frame_size) * 1/av_q2d (gdigrab-> time_base) * 8; return 0; error: // if an error occurs, if (source_hdc) ReleaseDC (hwnd, source_hdc); if (dest_hdc) DeleteDC (dest_hdc); if (hbmp) DeleteObject (hbmp); if (source_hdc) deleteDC (source_hdc); return ret ;}

From the source code, we can see that the gdigrab_read_header () process is roughly as follows:
(1) determine the handle hwnd of the window. If "title =" is specified, call FindWindow () to obtain hwnd. If "desktop" is specified, the hwnd is set to NULL.
(2) determine the rectangular area of the captured Screen Based on the Windows handle hwnd. If the specified window is crawled, The GetClientRect () function is used; otherwise, the entire screen is captured.
(3) Call the gdi api to complete screen capture initialization. Including:

A) Use GetDC () to obtain the HDC of a window handle (source_hdc here ).
B) Use CreateCompatibleDC () to create an HDC compatible with the specified device (here dest_hdc)
C) Create HBITMAP through CreateDIBSection ()
D) bind HBITMAP and HDC through SelectObject () (dest_hdc)

(4) Create an AVStream through avformat_new_stream.
(5) Save some parameters during initialization to the Context Structure of GDIGrab.

Gdigrab_read_packet () is used to read screen capture data from a frame. The function is defined as follows.

/*** Grabs a frame from gdi (public device demuxer API ). ** @ param s1 Context from avformat core * @ param pkt Packet holding the grabbed frame * @ return frame size in bytes */static int gdigrab_read_packet (AVFormatContext * s1, AVPacket * pkt) {struct gdigrab * gdigrab = s1-> priv_data; // read parameter HDC digest = gdigrab-> digest; HDC source_hdc = gdigrab-> source_hdc; RECT digest = gdigrab-> digest; AVRational time_base = bytes-> time_base; int64_t time_frame = gdigrab-> time_frame; BITMAPFILEHEADER bfh; int file_size = bytes-> header_size + bytes-> frame_size; int64_t curtime, delay; /* Calculate the time of the next frame */time_frame + = INT64_C (1000000);/* Run Window message processing queue */if (gdigrab-> show_region) gdigrab_region_wnd_update (s1, gdigrab);/* wait based on the fram E rate * // latency for (;) {curtime = av_gettime (); delay = time_frame * av_q2d (time_base)-curtime; if (delay <= 0) {if (delay <INT64_C (-1000000) * av_q2d (time_base) {time_frame + = INT64_C (1000000);} break;} if (s1-> flags & AVFMT_FLAG_NONBLOCK) {return AVERROR (EAGAIN);} else {av_usleep (delay) ;}// create an AVPacket if (av_new_packet (pkt, file_size) <0) return AVERROR (ENOMEM ); pkt-> pts = cur Time;/* Blit screen grab * // key: BitBlt () screen capture function if (! BitBlt (dest_hdc, 0, 0, clip_rect.right-direction, reverse-direction, source_hdc, direction, direction, SRCCOPY | CAPTUREBLT) {WIN32_API_ERROR ("Failed to capture image "); return AVERROR (EIO);} // draw the mouse pointer? If (gdigrab-> draw_mouse) paint_mouse_pointer (s1, gdigrab);/* Copy bits to packet data * // BITMAPFILEHEADER bfh In the BMP file header. bfType = 0x4d42;/* "BM" in little-endian */bfh. bfSize = file_size; bfh. bfReserved1 = 0; bfh. bfReserved2 = 0; bfh. bfOffBits = gdigrab-> header_size; // copy data to AVPacket // copy BITMAPFILEHEADER memcpy (pkt-> data, & bfh, sizeof (bfh )); // copy BITMAPINFOHEADER memcpy (pkt-> data + sizeof (bfh), & gdigrab-> bmi. bmiHeader, sizeof (gdigrab-> bmi. bmiHeader); // uncommon if (gdigrab-> bmi. bmiHeader. biBitCount <= 8) GetDIBColorTable (dest_hdc, 0, 1 <gdigrab-> bmi. bmiHeader. biBitCount, (RGBQUAD *) (pkt-> data + sizeof (bfh) + sizeof (gdigrab-> bmi. bmiHeader); // copy the pixel data memcpy (pkt-> data + gdigrab-> header_size, gdigrab-> buffer, gdigrab-> frame_size); gdigrab-> time_frame = time_frame; return gdigrab-> header_size + gdigrab-> frame_size ;}

From the source code, we can see that the gdigrab_read_packet () process is roughly as follows:
(1) read the parameters set during initialization from the GDIGrab context struct.
(2) Delay Based on the frame rate parameter.
(3) Use av_new_packet () to create an AVPacket.
(4) use BitBlt () to capture the screen.
(5) If you need to draw the mouse pointer, call paint_mouse_pointer (). No analysis is performed here.
(6) copy the following three items to the memory pointed to by AVPacket data in sequence:

A) BITMAPFILEHEADER
B) BITMAPINFOHEADER
C) pixel data from captured screens

Gdigrab_read_close () is used to disable gdigrab. The function is defined as follows.

/** * Closes gdi frame grabber (public device demuxer API). * * @param s1 Context from avformat core * @return 0 success, !0 failure */static int gdigrab_read_close(AVFormatContext *s1){    struct gdigrab *s = s1->priv_data;    if (s->show_region)        gdigrab_region_wnd_destroy(s1, s);    if (s->source_hdc)        ReleaseDC(s->hwnd, s->source_hdc);    if (s->dest_hdc)        DeleteDC(s->dest_hdc);    if (s->hbmp)        DeleteObject(s->hbmp);    if (s->source_hdc)        DeleteDC(s->source_hdc);    return 0;}

From the source code, we can see that gdigrab_read_close () has completed cleaning of various variables.

Lei Xiaohua
Leixiaohua1020@126.com
Http://blog.csdn.net/leixiaohua1020