Overview of DirectShow Development Based on Delphi

Part 1: Background Knowledge
DirectShow is a set of development kits provided by Microsoft for streaming media processing on the Windows platform and released together with the DirectX Development Kit. It evolved from DirectX Media in DirectX 6.0 and integrated with other members of the DirectX family (DirectDraw, directsound, etc.). It can be said to be a "integrator ".
What can DirectShow do? DirectShow provides powerful support for capturing and playback of Multimedia Streams. Using DirectShow, you can easily capture data from a data acquisition card that supports the WDM driver model, and perform corresponding post processing or even storage to files. It supports a wide range of media formats, including ASF, MPEG, Avi, DV, MP3, and wave, making it easy to play back multimedia data. In addition, DirectShow directly supports playback of DVDs, non-linear editing of videos, and data exchange with digital cameras. More importantly, DirectShow provides an open development environment. Each function module adopts the COM component method called filter, developers can also develop their own function filters to expand DirectShow applications. Based on functions, filters are divided into three categories: Source Filter, transform filter, and rendering filter. The former is responsible for obtaining data, the data source can be a file, a digital camera, and then the data is transmitted down; the intermediate is responsible for data format conversion, such as data stream separation and synthesis, encoding and decoding, etc, then, the data is further transmitted. The latter is responsible for the data destination-play or output to the sound card or video card for file storage.

Part 2 core technologies
The development of DirectShow is actually the development of filter, which is provided by DirectShow itself. The following is an overview of filter.
1. DirectShow Filter
Filters are generally divided into the following types.
(1) Source Filter: The Source Filter imports data into the filter chart. The data source can be files, networks, cameras, etc. Different Source filters process different types of data sources.
(2) transform filter: the function of the transform filter is to obtain the input stream, process the data, and generate the output stream. The conversion filter processes data by encoding/decoding, format conversion, compression, and decompression.
(3) Renderer Filter: The submit filter is at the last level in the filter chart. They receive data and submit the data to the peripherals.
(4) splitter filter: The splitter filter divides the input stream into multiple outputs. For example, the AVI split filter splits an AVI byte stream into a video stream and an audio stream.
(5) MUX filter: A hybrid filter combines multiple inputs into a separate data stream. For example, the AVI hybrid filter combines video streams and audio streams into an AVI byte stream.
These types of filters are not absolute. For example, an ASF reader filter is both a source filter and a segmentation filter.
2. filter graph Manager
Filter graph manager is also a COM object used to control all filters in filter graph. It has the following functions:
1) it is used to coordinate the state changes between filters, so that the State Changes of all filters in graph should be consistent.
2) create a reference clock.
3) return the filter Message notification to the application. Program
4) provides a method to create a filter graph.
Graph is a flow chart composed of various filters.
Sourcefilter -- | -- spliterfilter ----- (Video-pin)> -- transformfilter-> videorender
| ------- (Audio-pin)-> -- acmwraperfilter-> directsoundfilter
When the program starts, it first creates the COM Object of each filter, then adds it to the graph using filtergraph. addfilter, and then
Data Streams connect outpin and inpuin. It is best to start filtergraph. Play.
DirectShow is based on modularization. Each function module adopts the COM component method, which is called filter. DirectShow provides a series of standard modules for application development. developers can also develop their own function filters to expand DirectShow applications. The following example shows how to use a filter to play an AVI video file.
1) read Avi Data from a file to form a byte stream. (This task is done by the source filter)
2) Check the Header Format of the Avi Data Stream and separate the video stream from the audio stream using the AVI split filter.
3) decode the video stream and select different decoder Filters Based on different compression formats.
4) redraw the video image using the Renderer Filter.
5) audio streams are sent to the sound card for playback. The default directsound device filter is generally used.
When the status changes, the filter status changes in graph should be consistent. Therefore, the application directly sends the state change command to the filter, but sends the corresponding state change command to the filter graph manager, the manager distributes commands to each filter in the graph. Seeking also works in the same way. First, the application sends the seek command to the filter graph manager and distributes it to each filter.
Reference clock: the filter in graph uses the same clock, which is called reference clock. The reference clock ensures that all data streams are synchronized, the time when the video watermark or audio watermark should be submitted is called the presentation time. the presentation time is determined relative to the reference clock. Filter graph manager should select a reference clock. You can select the clock on the sound card or the system clock.
Graph event. Graph Manager uses the event mechanism to notify applications of events in graph. This mechanism is similar to the Message Loop Mechanism in windows.
Graph construction method. The graph manager provides the application with the method of adding the filter to the graph, connecting the filter method, and disconnecting the filter method.
However, the graph manager does not provide a way to send data from one filter to another. This is done independently by the filter via pin internally.
3. Media type
Because DirectShow is based on the COM component, You need to describe the data format of each vertex of the filter graph. For example, we also take the playback of the AVI file as an example, the data enters the graph in the form of riff blocks and is then divided into video and audio streams. The video stream consists of a series of compressed video clips. After decompression, the video stream consists of a series of uncompressed bitmaps, and the audio stream follows the same steps.
Media types are common and can be extended to describe digital media formats. When two filters are connected, they will agree on a media type. The media type defines the data that the filter at the source will send to the downstream filter, and the physical layout of the data. If two filters cannot support the same media type, they cannot be connected.
For most applications, you may not need to consider the media type. However, in some applications, you will apply the media type directly.
The media type is defined through the am_media_type structure.
Filters transmits data through the pin connection. The data stream is the input pin from the output pin of a filter to the connected filter. The commonly used method for outputting pin data is to call the imeminputpin: receive method on the input pin.
For the filter, there are several ways to allocate memory blocks used by media data, which can be allocated on the stack, on the DirectDraw surface, or using the shared memory of GDI, there are other methods. In DirectShow, the memory allocation task is Allocator, which is also a COM Object and exposes an imemallocator interface.
When two pins are connected, one pin must provide an allocator. DirectShow defines a series of function calls to determine which pin provides Allocator, and the number and size of buffer.
Before the data stream starts, Allocator creates a memory pool of buffer. After the data stream is sent, the source filter fills the data into an idle buffer in the memory pool, and then pass it to the following filter. However, the source filter does not directly pass the memory buffer pointer to the downstream filter, but uses a media samples COM Object. This sample is created by allocator to manage the memory buffer. Media sample exposes the imediasample interface. A sample contains the following content:
A pointer to the memory that is not sent.
A timestamp
Some logos
Media type.
The timestamp indicates the presentation time. The Renderer Filter schedules the render order based on the time. A flag is used to indicate whether the data is interrupted. A media type provides a method to change the data format midway through. However, generally, the sample has no media type, indicating that their media type has not changed.
When a filter is using a buffer, it will keep the reference count of a sample. Allocator uses the reference count of the sample to determine whether a buffer can be reused. This prevents buffer usage conflicts. When all filters release a reference to the sample, the sample is returned to the memory pool of Allocator for reuse.
Majortype: main type, such as video, audio, or bitstream
Subtype: Auxiliary description type, such as yuv12 or uyvy in the video.
Formattype: format description, more detailed structure. For example, video size, frequency, frame rate, etc,
You can use the format_videoinfo (videoinfoheader) and format_waveformatex (waveformatex) struct to describe
// Pammediatype
When the am_media_type data structure is used to describe the media type, if both majortype, subtype, and formattype specify guid
This is the complete media type.
* ************************************ Filter connection ************************************
The filter connection is actually a pin connection. The connection direction is always directed by the output pin of the filter (upstream filter) at the upper level to the lower level.
Filter (downstream filter) Input Pin.
1. Filter connection process
Pin is also a COM interface. Implements the ipin interface. Generally, you can call (the following functions are connected ):
Ifiltergraph. connectdirect, igraphbuilder. Connect, igraphbuilder. Render, igraphbuilder. renderfile
{The following is an example. Generally, the filter is connected when it is stopped.
// Connect source-> mpeg1spliter
Source. findpin (stringtoolestr ('output'), outpin );
Mpeg1splitter. findpin (stringtoolestr ('inpin'), inpin );
HR: = (filtergraph1 as igraphbuilder). Connect (outpin, inpin );
If failed (HR) then begin
Showmessage ('failed connect MPG source-> mpeg1splitter ');
Exit;
End;
}
2. method for constructing filtergraph
1) ifiltergraph. addfilter: this parameter provides a filter object and adds it to filtergraph.
2) ifiltergraph. connectdirect: this parameter provides output pin, input pin, and media type for direct connection.
3) igraphbuilder. addsourcefilter: this parameter provides the source file name and automatically loads a sourcefilter to filtergraph.
4) igraphbuilder. Connect this parameter provides the output pin, input the pin, and media type for connection. If the connection fails, it will automatically try to add the necessary format conversion filter in the middle.
5) igraphbuilder. render this parameter provides the output pin, and automatically adds the necessary filter to build the remaining part of the filtergraph (until it is connected to the renderfilter)
6) igraphbuilder. Render: this parameter provides the source file name and automatically adds necessary filters to the file for playback.
{
// The following example indicates that there are 6 filters in the filtergraph. They are all created from COM objects.
VaR
Source: ibasefilter;
Mpeg1splitter: ibasefilter;
Mpegvcodec: ibasefilter;
Avidec: ibasefilter;
Avidest: ibasefilter;
Writer: ibasefilter;
HR: hresult;
Outpin, inpin: ipin;
Begin
Cocreateinstance (clsid_asyncreader, nil, clsctx_inproc, iid_ibasefilter, source); // typical pull mode
Cocreateinstance (clsid_mpeg1splitter, nil, clsctx_inproc, iid_ibasefilter, mpeg1splitter); // mpeg1 format
Cocreateinstance (clsid_cmpegvideocodec, nil, clsctx_inproc, iid_ibasefilter, mpegvcodec); // MPEG Encoding
Cocreateinstance (clsid_avidec, nil, clsctx_inproc, iid_ibasefilter, avidec); // Avi Decoding
Cocreateinstance (clsid_avidest, nil, clsctx_inproc, iid_ibasefilter, avidest); // Avi target
Cocreateinstance (clsid_filewriter, nil, clsctx_inproc, iid_ibasefilter, writer); // write the file
(Filtergraph1 as ifiltergraph). addfilter (source, 'source ');
(Filtergraph1 as ifiltergraph). addfilter (mpeg1splitter, 'mpeg1splitter ');
(Filtergraph1 as ifiltergraph). addfilter (mpegvcodec, 'mpegvcodec ');
(Filtergraph1 as ifiltergraph). addfilter (avidec, 'avidec ');
(Filtergraph1 as ifiltergraph). addfilter (avidest, 'avidest ');
(Filtergraph1 as ifiltergraph). addfilter (writer, 'writer ');
End;
}
3. Generally, graphedit can be used to view the filters installed normally in the system. If they are installed in the DirectShow directory, you can specify CLSID
Use cocreateinstance to create the instance. In other directories, it must be created through system enumeration.
The system provides a clsid_systemdeviceenum, which is created using cocreateinstance and obtains the icreatedevenum interface. Then
Use icreatedevenum. createclassenumerator to create an enumerator for the specified directory type and obtain the ienummoniker interface.
The ienummoniker. Next method is used to media all the available device IDS (device moniker) in this directory. The imoniker interface is implemented for each device ID object.
After calling imoniker. bindtostorage, you can access the property set of the device identity. For example, get the display name of the device.
Call imoniker. bindtoobject to bind the device ID to a direcshowfilter, and then call ifiltergraph. addfilter to add filtergraph.
Participate in the work.
DirectShow usually has two names: Display name: @ device: Cm: {33d9a760-90c8-11d0-bd43-00a0c911ce86} \ Xvid
Friendly name: Xvid MPEG4 Decoder
{The following is the call Code
VaR
I, J: integer;
Amoniker, mymoniker: imoniker;
Propbag: ipropertybag;
Avariant: olevariant;
CLSID: tguid;
Found: Boolean;
Begin
For I: = 0 to sysdevenum. countcategories-1 do
Cbcategories. Items. Add (sysdevenum. categories [I]. friendlyname); // sysdevenum: tsysdevenum;
Found: = false;
J: = 0;
Mymoniker: = filter. basefilter. moniker;
If mymoniker = nil then exit;
Mymoniker. bindtostorage (nil, nil, ipropertybag, propbag );
If propbag. Read ('clsid ', avariant, nil) = s_ OK then
CLSID: = stringtoguid (avariant)
Else CLSID: = guid_null;
For I: = 0 to sysdevenum. countcategories-1 do
Begin
Sysdevenum. selectindexcategory (I );
If sysdevenum. countfilters> 0 then
For J: = 0 to sysdevenum. countfilters-1 do
Begin
If isequalguid (CLSID, sysdevenum. Filters [J]. CLSID) then
Begin
Amoniker: = sysdevenum. getmoniker (j );
Found: = amoniker. isequal (mymoniker) = s_ OK;
Amoniker: = nil;
End;
If found then break;
End;
If found then
Begin
Cbcategories. itemindex: = I;
Cbcategorieschange (NiL );
Lbfilters. itemindex: = J;
Lbfiltersclick (NiL );
Break;
End;
End;
Propbag: = nil;
Mymoniker: = nil;
}
Pammediatype = ^ tammediatype;
_ Ammediatype = record
Majortype: tguid;
Subtype: tguid;
Bfixedsizesamples: bool;
Btemporalcompression: bool;
Lsamplesize: ulong;
Formattype: tguid;
Punk: iunknown;
Cbformat: ulong;
Pbformat: pointer;
End;
Filter development basics-Basic Analysis
1) tbcbasefilter
Tbcbasefilter = Class (tbcunknown, ibasefilter, iamoviesetup)
Is the base class of the most basic filter. Usage:
(1) declare a new class to inherit from tbcbasefilter
(2) define a filter pin instance in the new class. (PIN inherited from tbcbasepin)
(3) Implement the pure virtual function tbcbasefilter. getpin, which is used to return the object pointer of each pin on the filter.
(4) Implement the pure virtual function tbcbasefilter. getpincount to return the number of pins on the filter.
(5) Consider how to process the sample data from the input pin.
2) tbcbasepin
Tbcbasepin implements the pin interface, and tbcbasepin designs the entire connection process of the pin. The iqualitycontrol quality control interface is also implemented.
Three member functions are implemented on tbcbasepin to correspond to the filter status.
(1) filter. Stopped <-----------> tbcbasepin. inactive
(2) filter. spaused <-------------> tbcbasepin. Active
(3) filter. Running <-------------> tbcbasepin. Run
When developing a filter, you may rewrite these three functions to initialize and release necessary resources. Implementation Method:
(1) derive a subclass from tbcbasepin
(2) Implement the pure virtual function tbcbasepin. checkmediatype to detect the media type During pin connection.
(3) Implement the pure virtual function tbcbasepin. getmediatype to provide the first media type on the pin.
(4) Implement ipin. beginflush and ipin. endflush Functions
(5) functions that may need to be rewritten include. Tbcbasepin. inactive, tbcbasepin. Active, tbcbasepin. Run,
Tbcbasepin. checkconnect (check when connecting, for example, querying whether a special interface is supported on the pin of the other party ),
Tbcbasepin. breakconnect (disconnect and release necessary resources ),
Tbcbasepin. completeconnect (called when the connection is complete. You can obtain the media type and other parameters for the current connection in this function ),
Tbcbasepin. endofstream (called when all the upstream data is transferred. If this is a transformfilter, it will continue to be sent,
If an renderfilter is used, an ec_complete event must be sent to filtergraph)
Tbcbasepin. Y (directly responds to quality control, or sends quality control messages to the filter of the previous level)
3) tbcbaseinputpin and tbcbaseoutputpin
Both tbcbaseinputpin and tbcbaseoutputpin are derived from tbcbasepin,
Tbcbaseinputpin implements imeminputpin (used for data transmission in Push mode)
Tbcbaseoutputpin completes the negotiation of the sample Manager (allocate) used for data transmission, and overwrites tbcbasepin. Active (used
Actual sample memory allocation), tbcbasepin. inactvie (used for the release of sample memory ).
Tbcbaseinputpin usage (a subclass is derived and at least the following function is rewritten ):
(1) tbcbaseinputpin. beginflush
(1) tbcbaseinputpin. endflush
(1) tbcbaseinputpin. Receive
(1) tbcbaseinputpin. checkmediatype (this function is generally implemented on the output pin)
(1) tbcbaseinputpin. getmediatype
Tbcbaseoutputpin usage (a subclass is derived and at least the following function is rewritten ):
(1) rewrite tbcbasepin. checkmediatype to check the media type during connection.
(2) Implement the pure virtual function tbcbaseoutputpin. decidebuffersize to determine the size of the sample memory.
(3) rewrite the pure virtual function tbcbasepin. getmediatype to provide the first media type on the pin.
4) tbcmediatype
Tbcmediatype is the implementation class of the sample used for data transmission. tbcmediatype implements the imediasample2 interface, and tbcmediatype encapsulates
A pointer pointing to a piece of memory, obtained through tbcmediatype. getpointer.
This article source on the iceberg podcast http://xinsync.xju.edu.cn/, original address: http://xinsync.xju.edu.cn/index.php/archives/986