Media objects of DirectX

Microsoft's DirectX Media object (DMOs) is a new method for writing streaming data. For some reason, DMOs are very similar to Microsoft's DirectShow filter. DMO processes data by using input data to generate output data. DMO is more similar to DirectShow than API functions. Therefore, DMO is very easy to create, test, and use. DMO is fully compatible with DirectShow, but applications can use DMO without DirectShow. In the DirectShow application, DMO is encapsulated by the DirectShow filter, which is called the DMO encapsulation filter. The difference between the filter and DMO lies in the transparency of the application, that is, the dmo api is not required, that is, the application can directly use DMO without DirectShow. Therefore, writing a DMO will create a component that can be widely used in applications. Differences between DMO and DirectShow filters only work in DirectShow image filtering. In images, the filter image manager acts as a medium between applications and filters. DMO does not have this requirement. A single application can use DMO independently. In DirectShow, filters perform a lot of work on streaming data. Including: Allocation buffer. · Ruling media types and connecting to other filters · filter image pressure stack data · send events to the filtering graphics manager · synchronize multiple threads to compare, a DMO does not do anything above. Instead, the customer uses these DMO types. The customer allocates buffering for them and fills in the data. Then the buffer is passed to DMO, and DMO will process them. The customer obtains the output buffer. In DirectShow, the DMO encapsulation filter is a customer of DMO, so it can process all these tasks, while other applications can provide their own application instances. DMOs has the following advanced features: · There is little function support, so it is usually smaller and simpler than the DirectShow filter. · Because the image filtering is not required, it is more flexible than DirectShow. You can use this service when you need the services provided by DirectShow, such as instance synchronization, smart connection, automatic processing of streaming data, and thread management. If you do not need these services, you can directly access DMO. · DMOs can often process synchronized data, which avoids the multi-thread issue that filter developers must consider. · Unlike ACM and VCM decoding, DMO is based on the Component Object Model and can be extended through the QueryInterface interface. · DMOs supports more irregular stream modes than ACM and VCM decoding. Like DirectShow filter, DMOs also supports multiple inputs and outputs. For the above reasons, we should try to use DMO instead of DirectShow. However, in some cases, DMO is not suitable. Example: · abnormal transmission requirements. DMO encapsulation filters are transmitted using imeminputpin. If you need to use another transmission mechanism in the DirectShow application, you must write a new filter. · Create a dynamic image. If the application executes Dynamic Graph creation, you may need to write your own filter. For example, you need to control the creation of the allocation program. The DMO test program directx8.0 SDK includes a DMO test program. The name is dmotest. This test program can help test DMO. This program is installed in the bin/dxutils directory of the SDK. If you want to develop DMO, the DMO test program will be part of the test. To use dmotest, a test data must be generated for DMO. Therefore, the SDK contains a DirectShow filter in dmodump. dll, which must be registered with regsvr32. The section "use DMO media objects" describes how to use DMO objects in an application. Here we will only describe how to directly access DMOs. The following section describes how to use DMO in DirectShow. This section contains the following parts. · Stream and buffer · Data Stream · how to process data · discarded and selectable stream · internally processed DMOs stream and buffered DMO has m input to generate N output objects. Both input and output are called streams. Each DMO must have at least one stream. A DMO can have no input stream or no output stream, but a typical DMO includes input and output streams. Note: The stream is not a clear COM object. They are references of DMO objects and are indexes starting from 0. A DMO obtains data through an input stream. It first processes the data and then outputs the data through the output abortion. All data types Use one media type ). Media types define how to identify data content. For example, a 24-bit RGB video of 320x240 is a type, and a PCM audio of 44.1Hz and 16-bit is another type. Describe the media type using the dmo_media_type structure. Each stream in a DMO can receive a range of media types. DMO can be used to receive video streams of any large bit or videos of only 16 bits. A DMO can be restricted to the input and output of a specific link. For example, if an input stream is set to a 16-bit video, the corresponding output may require the same bit. The application can enumerate the preferred media type of each stream and test the specified input/output Association. For example, a DMO that creates Avi Data can have two input streams: video and audio. It may generate a data stream for cross-access to audio and video. · A multi-tone and color-separation RGB image may have 24-bit RGB video input and 8-bit RGB video output. The input data submitted by the application is divided in the memory block. Each memory block is encapsulated by a COM object called a buffer. This buffer must support the imediabuffer interface. This interface contains a method to set the buffer Data Length, receive Data Pointer, and receive the buffer allocation size. The application can use this method to allocate all the buffers, which is applicable to both input and output. This section describes how to move data in applications and DMO. The application alternate delivery input buffering and request output. Processing input buffering can call the imediaobject: processinput method in DMO. You can call the imediaobject: processoutput method to process the output buffer. There are two ways to stop the input data stream. · Interruption mode: When an application enters data, or the data is interrupted, the application terminates with a signal. After DMO is interrupted, no input is generated. At the same time, DMO can still process data. Therefore, the application should request the output until the DMO notification has no data available. · Flushing method: applications can use flushing to interrupt data streams. If DMO has data in the buffer, the data will be immediately released. The stream starts when the application calls imediaobject: processinput for the first time. However, dmo does not start streaming until the application sets the media type for each stream. (Except for non-mandatory streams) when the application notifies DMO in a rinse mode, or notifies each stream of data in an interrupt mode and then processes all the output, the stream data will be stopped. These actions will cause DMO to return to a non-stream state. DMO retains all media type settings, but releases all imediabuffer pointers. There will be no output in the future until the application calls processinput again. To process streaming data, follow these steps. 1. query DMO to obtain the number of streams and the media type of each stream. 2. Set the media type for all streams. 3. Allocate input buffer and output buffer. 4. Call processinput to fill the input buffer. 5. Call processoutput to receive output data. Repeat 4 and 5 until all input data is processed. 6. Send an interrupt signal and process the remaining output. Steps 4 to 6 can be interrupted by flushing data. This section describes how to process data in detail. Step 1 query DMO first, query DMO to obtain the number of streams and the media type of each stream. To obtain the number of input and output streams, you can call the imediaobject: getstreamcount method. For each stream, DMO will sort the priority of the preferred media type, and then assign an index for each type. The index starts from 0. To obtain the details of a preferred media type, you can call the imediaobject: getinputtype method to process the input. The output can call the imediaobject: getoutputtype method. To enumerate all media types in a stream, you can use the cyclically increasing media type index until the method returns dmo_e_no_more_items. The following is the Demo code.

DWORD cInputs, cOutputs, type = 0

DMO_MEDIA_TYPE mt

pDMO->GetStreamCount(&cInputs, &cOutputs)

for (DWORD i = 0; i < cInputs; i++)

{

    while (pDMO->GetInputType(i, type, &mt) != DMO_E_NO_MORE_ITEMS)

    {

        if ( this media type is one you want )

            break

        MoFreeMediaType(&mt)

        type++

    }

}

The getinputtype and getoutputtype Methods return a dmo_media_type structure. The following figure shows the corresponding structure members. · Primary type: guid of the specified streaming media type. The main type is a general classification, like video and audio. · Sub-type: a guid of the specified stream sub-type. Subtypes are more detailed categories. For example, mediasubtype_rgb24 2 specifies a 24-bit RGB video. · Pbformat: A pointer pointing to the format description in the structure. The information specified in the format structure, for example, the video width or height, or an audio rate. Different media types use different formats. Most video data uses the videoinfoheader structure. The audio data adopts the waveformatex structure. · Formattype: Specify the Format Structure Containing pbformat. The media type may have a null Format Structure, indicating that formattype is the value of guid_null. A null format indicates that DMO can accept a certain range of formats within the specified media type. For example, a stream requires the format of PCM audio and Its similar sampling rate range. Therefore, it will return mediatype_audio as the main type, the mediasubtype_pcmaudio sub-type in the null format. The application must call the mofreemediatype function to release the pbformat. Step 2. Set the media type. After the DMO primary media type is found, you can call the imediaobject: setinputtype and imediaobject: setoutputtype methods to set the media type for each stream. DMO cannot guarantee that each reported media type combination is valid. Just as the output type may need to match the input type. You can test a media type and use the dmo_set_typef_test_only flag to call setinputtype or setoutputtype. For decoder, you should set the input type and select an output type. For encoder, you should set the output type and select an input type. Because setting a stream will affect other streams, you need to clear the previously set media type. This function can call setinputtype or setoutputtype to fill the dmo_set_typef_clear flag with the parameter. Step 3 assign a buffer after setting the media type, each buffer request queries DMO. These can be changed by media types. For each stream, you can call the mediaobject: getinputsizeinfo or imediaobject: getoutputsizeinfo method. These methods return three values: · minimum buffer size, in bytes. If a buffer is aligned and the start address is a multiple of a specified integer, in these cases, alignment is required. · For some input buffering, the maximum data volume is controlled by DMO. The input data is a buffer multiple. Sufficient buffer must be allocated to process these requests. To meet the dimensional requirements, an input stream may require each buffer to contain a unique and complete sample, or contain an exact sample, or use a fixed sampling size. To meet these requirements, you can call the imediaobject: getinputstreaminfo method. Step 4. process the input and then deliver an input buffer to DMO. Fill in one or more input buffers with media data for each input stream. You can directly write data to the buffer, or use an output buffer from another DMO. Call the processinput method to deliver each buffer. When the application flushes DMO, the buffer can be released, but do not reuse the buffer before the DMO is released. To determine whether a stream can receive more data, you can call the imediaobject: getinputstatus method. If the stream can accept more data, this method returns the dmo_input_statusf_accept_data flag. Step 5 process the output because processinput delivers an inbound/outbound buffer at a time, processoutput usually outputs all the stream data at a time. The application passes a dmo_output_data_buffer matrix, which is used for each output stream. This structure contains a pointer pointing to the output buffer and different information filled. In the processoutput method, dmo generates as much data as possible (the specified output buffer size ). If the output buffer is filled before processing all the data, it will set the dmo_output_data_bufferf_incomplete flag in the dwstatus structure. When the method returns, the flag of each structure is checked. If the flag exists, call processoutput again. After the stream starts, DMO can always receive data, generate output data, or both. Therefore, it is not the dmo_input_statusf_accept_data flag returned by getinputstatus, that is, the dmo_output_data_bufferf_incomplete flag returned by processoutput. The application maintains the data flow by testing these flags and calling processinput or processoutput. Step 6 send an interruption signal when all valid input data is delivered, call the imediaobject: Discontinuity Method to end. A DMO of the discarded and selectable (non-mandatory) streams can specify some output streams as the discarded or optional streams: · A discarded stream includes data that can be discarded by the application, complete or intermittent sampling. · An optional stream is a secondary stream not required by all applications, or a stream that includes mainstream additional information. To query whether a stream is discarded or optional, you can call the imediaobject: getoutputstreaminfo method and check the pdwflags parameter. The dmo_output_streamf_discardable flag indicates that a stream can be discarded. The dmo_output_streamf_optional flag indicates that the stream is selectable. Generally, at least one stream is not selectable. When calling the imediaobject: processoutput method, you can discard data by setting the dmo_process_output_discard_when_no_buffer flag in the dwflags parameter from one or more streams. For each stream to be discarded, The pbuffer of the dmo_output_data_buffer structure must be set to null. If pbuffer is NUL but the dmo_process_output_discard_when_no_buffer flag is not set, DMO will not discard the data, even if the stream can be discarded or selectable. If pbuffer is null but no data is discarded, dmo sends a dmo_output_data_bufferf_incomplete flag to indicate that the stream still has data. If an output stream is not processed but is not discarded, DMO cannot generate data in other output streams. If you never use an optional stream, you do not need to set its media type. In the same case, it is not suitable for discarded streams. The media type must be set in a discarded stream. In-Place media objects some data transformations can be directly performed by modifying data to accomplished, which is called In-Place Processing. Many audio and video effects can be processed in proper locations. In-Place Processing is more effective than directly copying data to another buffer. To process in-place data, you can independently call the imediaobjectinplace: process method instead of separately calling processinput and processoutput. Transmits a byte array containing input data. When the method returns, the byte array contains the output data. All mediaobject methods must be supported for imediaobjectinplace. You can use in-place to process or create an independent input/output buffer. Do not confuse the two types of processing. If you call process, do not call processinput or processoutput, and vice versa. After the input is stopped, in-place DMO can create some additional output, namely, the end effect (Effect Tail ). For example, the reverb effect continues to work after the input is "quiet. DMO produces the final effect. The program must use the zero-input buffer to call the process method until the final result is processed by completely.