Quickly render video textures using Ogre

Basic ideas

The full-screen quad rectangle2d with the same size as the video screen can be used for testing, which uses a dynamic texture material.

Dynamically replace the texture unit of the material with the current frame image at render time by frame rate

Video Read

Read each frame of video I am using OPENCV, similar to the following:

cvcapture* mcapture = Cvcreatefilecapture (Mfilename.c_str ());

int totalframes = (int) cvgetcaptureproperty (mcapture, Cv_cap_prop_frame_count);

int fps = Cvgetcaptureproperty (mcapture, Cv_cap_prop_fps);

iplimage* frame = Cvqueryframe (mcapture);

Frame is the captured picture, where the three most needed properties are:

Width image Wide

Height Image Height

ImageData data area in RGB format

You can use any data source, the final ogre processing is the same

First attempt

The first thought is to use the existing material directly, by unloading and loading the texture unit:

Unload the original texture

Ogre::materialptr mat = Ogre::materialmanager::getsingleton (). Getbyname ("Testmat");

Ogre::textureptr textureold = mat->gettechnique (0)->getpass (0)->gettextureunitstate (0)->_ Gettextureptr (0);

Textureold->unload ();//optional

Ogre::texturemanager::getsingleton (). Remove (static_cast<ogre::resourceptr> (textureold));

Loading new textures

Ogre::D atastreamptr pdatastream (New Ogre::memorydatastream (Frame->imagedata, frame->width * frame->height * 3, False, true));

Ogre::textureptr texture = Ogre::texturemanager::getsingleton (). Loadrawdata ("Testtexture",

Ogre::resourcegroupmanager::D efault_resource_group_name, Pdatastream, Frame->width, Frame->height,

Ogre::P f_r8g8b8, ogre::tex_type_2d);

Mat->gettechnique (0)->getpass (0)->gettextureunitstate (0)->settexturename ("TestTexture");

Because we have raw memory data frame->imagedata, we can use the memory data stream and call the Loadrawdata method to load the texture

This approach is supposed to be fast, so just worry about whether the right picture

After the operation, the screen is displayed, but the gray dodo, without the original gloss

Think about the need to enable gamma correction, that is, texture creation instead:

Ogre::textureptr texture = Ogre::texturemanager::getsingleton (). Loadrawdata ("Testtexture",

Ogre::resourcegroupmanager::D efault_resource_group_name, Pdatastream, Frame->width, Frame->height,

Ogre::P f_r8g8b8, ogre::tex_type_2d, 0, 1.0f, true);

The last three parameters, respectively, set Mipmap to 0, using hardware gamma correction (if the software correction, the 1.0f will be changed to 2.2, would be surprisingly slow)

This time, the image is exactly the same as the original video, but the frame rate is very low

Improved

Because each switch screen uses the same texture format, the size is exactly the same

It is obvious that you might consider replacing the memory data area of the existing texture unit directly instead of destroying it first and then creating the code as follows:

Ogre::P ixelbox Box (frame->width, Frame->height, 1, Ogre::P f_r8g8b8, Frame->imagedata);

Texture->getbuffer (0, 0)->blitfrommemory (box);

This time, it should be fast enough, but the results are still unsatisfactory.

Tracking down, found blitfrommemory similar implementations eventually call the D3dxloadsurfacefrommemory, the performance of the bottleneck is here

Because it's already a D3D API, this road can't go on anymore.

In addition, although the Hardwarepixelbuffer interface declares the WriteData interface, it is not implemented and cannot be used

There seems to be only one way to replicate the data through lock, and hopefully it will succeed:

void* DST = texture->getbuffer (0, 0)->lock (ogre::hardwarebuffer::hbl_discard);

memcpy (DST, Frame->imagedata, Frame->width * frame->height * 3);

Texture->getbuffer (0, 0)->unlock ();

In doing so, it turns out that the frame rate is actually up, but the image becomes a grayscale image of the ghosting, apparently because the memory format is inconsistent

Fortunately, the height of the displayed image is exactly the same as the original 3/4.

Consider that we are using the RGB format, perhaps because the hardware uses 4 bytes of RGBA to store pixels

The code is then modified to:

char* DST = (char*) (texture->getbuffer (0, 0)->lock (ogre::hardwarebuffer::hbl_discard));

char* src = frame->imagedata;

for (int i=0; i<frame->width; ++i)

{

for (int j=0; j<frame->height; ++j)

{

*dst++=*src++;

*dst++=*src++;

*dst++=*src++;

*dst++=0; Fill

}

}

Texture->getbuffer (0, 0)->unlock ();

Ah, this time it finally worked.

Further improvements

When using the default texture cell, the buffer purpose defaults to Tu_default

For scenes that need to be refreshed quickly, use tu_dynamic_write_only_discardable

You can therefore overwrite the creation of a texture with:

Texture = Ogre::texturemanager::getsingleton (). Createmanual (TextureName,

Ogre::resourcegroupmanager::D efault_resource_group_name, ogre::tex_type_2d, Frame->width, frame->height, 0,

Ogre::P f_r8g8b8, ogre::tu_dynamic_write_only_discardable, 0, true);

This can further improve performance

Quickly render video textures using Ogre