The layer-3 Texture of D3D is rendered by the pixel shader YUV420P and textureyuv420p

The layer-3 Texture of D3D is rendered by the pixel shader YUV420P and textureyuv420p

Briefly record some highlights of YUV420P rendering with Texture over the past two days.

During video playback, sometimes the decoded data is YUV420P. Surface can render YUV420P by setting parameters, but Texture does not seem to support direct rendering of YUV420P. Surface usage seems to be relatively simple. It is usually used to display data. If Texture is used, many other functions of D3D can be used for some effect. Of course, this depends on individual needs. Generally, it is enough to display video data on the surface.

1. Convert YUV420P data into RGB32 using the pixel coloring Tool

There is a fixed interval between frames during video playback. If the decoded data is YUV420P, You need to convert the data to RGB32 (it should be converted to RGB32, and no detailed investigation has been conducted, in the preceding example, the data used by the texture is RGB32. If this process is handed over to the CPU, the CPU usage will be increased, using GPU to do this operation releases some of the CPU pressure.

In this article, we use three layers of texture to store the Y, U, and V components of YUV420P (this word does not know whether to use it or not ), the pixel coloring er then computes the data of the three components into the data of RGB32 and then displays it. This is the core content of this article.

The HLSL code of the pixel shader is as follows:

sampler YTex;sampler UTex;sampler VTex;struct PS_INPUT{    float2 y    : TEXCOORD0;    float2 u    : TEXCOORD1;    float2 v    : TEXCOORD2;};float4 Main(PS_INPUT input):COLOR0{          float y = tex2D(YTex,input.y).r;    float u = tex2D(UTex, input.u.xy / 2).r  - 0.5f;    float v = tex2D(VTex,input.v.xy / 2).r  - 0.5f;                                float r = y + 1.14f * v;    float g = y - 0.394f * u - 0.581f * v;    float b = y + 2.03f * u;                    return float4(r,g,b, 1);}

The HLSL code can be directly written in a txt file. sampler can be used to identify the texture layer and sampling level objects. Direct3D will uniquely associate each sampler object with a texture layer. For more information about the specific HLSL syntax, check the information on your own. I also have a rough idea of what is going on. In the code, you can call the D3DXCompileShaderFromFile function to compile the pixel shader from the file. But in fact, I personally do not like this practice of placing code in a single file. Such code should be compiled into the exe as much as possible. However, I have only a preliminary understanding of D3D, but I don't know how to compile it into the exe. If anyone knows, I 'd like to advise.

ID3DXBuffer* shader      = 0;ID3DXBuffer* errorBuffer = 0;hr = D3DXCompileShaderFromFile(    "ps_multitex.txt",    0,    0,    "Main", // entry point function name    "ps_2_0",    D3DXSHADER_DEBUG,     &shader,    &errorBuffer,    &MultiTexCT);// output any error messagesif( errorBuffer ){    ::MessageBox(0, (char*)errorBuffer->GetBufferPointer(), 0, 0);    d3d::Release<ID3DXBuffer*>(errorBuffer);}if(FAILED(hr)){    ::MessageBox(0, "D3DXCompileShaderFromFile() - FAILED", 0, 0);    return false;}//// Create Pixel Shader//hr = Device->CreatePixelShader(    (DWORD*)shader->GetBufferPointer(),    &MultiTexPS);if(FAILED(hr)){    ::MessageBox(0, "CreateVertexShader - FAILED", 0, 0);    return false;}d3d::Release<ID3DXBuffer*>(shader);

In the preceding code, d3dxcompileshaderfromfile.pdf compiles the HLSL code from the file ps_multitex.txt. The Main parameter is the entry function of the HLSL code, as shown in the code above. This entry function can be customized, but must be consistent. ps_2_0 indicates the version of the pixel shader. The CreatePixelShader function creates a pixel shader.

2. Correlation Between sampler and Texture

Create a texture layer. In this article, the YUV420P rendering method uses three layers of texture, each layer of texture respectively stores Y, U, V data.

Device->CreateTexture ( Width, Height, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &YTex, NULL ) ;Device->CreateTexture ( Width / 2, Height / 2, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &UTex, NULL ) ;Device->CreateTexture ( Width / 2, Height / 2, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &VTex, NULL ) ;

Correlation between sampler and Texture

// // Get Handles//YTexHandle      = MultiTexCT->GetConstantByName(0, "YTex");UTexHandle        = MultiTexCT->GetConstantByName(0, "UTex");VTexHandle        = MultiTexCT->GetConstantByName(0, "VTex");//// Set constant descriptions://UINT count;MultiTexCT->GetConstantDesc(YTexHandle,      &YTexDesc, &count);MultiTexCT->GetConstantDesc(UTexHandle, &UTexDesc, &count);MultiTexCT->GetConstantDesc(VTexHandle,    &VTexDesc, &count);MultiTexCT->SetDefaults(Device);

Set the texture/sampler status. This part is made during rendering and can be directly written in HLSL code. The rendering part will also see the code, which is actually the same piece of code. I just cut it out from the rendering part in advance to express the overall process of texture association with sampler, hope it will not cause misunderstanding.

// Y texDevice->SetTexture(     YTexDesc.RegisterIndex, YTex);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);// U texDevice->SetTexture(     UTexDesc.RegisterIndex, UTex);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);// string texDevice->SetTexture(     VTexDesc.RegisterIndex, VTex);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);

3. Rendering YUV420P

Obtain the YUV420P data. YUV420P data read in this article.

Open the file code:

if((infile=fopen("test_yuv420p_320x180.yuv", "rb"))==NULL){    printf("cannot open this file\n");    return false;}

Read data and copy the data to the texture:

if (fread(buf, 1, Width*Height*3/2, infile) != Width*Height*3/2){        // Loop        fseek(infile, 0, SEEK_SET);        fread(buf, 1, Width*Height*3/2, infile);    }//// Render//Device->Clear(0, 0, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, 0xffffffff, 1.0f, 0);plane[0] = buf;plane[1] = plane[0] + Width*Height;plane[2] = plane[1] + Width*Height/4;D3DLOCKED_RECT d3d_rect;byte *pSrc = buf;//Locks a rectangle on a texture resource.//And then we can manipulate pixel data in it.LRESULT lRet = YTex->LockRect(0, &d3d_rect, 0, 0);if (FAILED(lRet)){    return false;}// Copy pixel data to texturebyte *pDest = (byte *)d3d_rect.pBits;int stride = d3d_rect.Pitch; for(int i = 0;i < Height;i ++){    memcpy(pDest + i * stride,plane[0] + i * Width, Width);}YTex->UnlockRect(0);D3DLOCKED_RECT d3d_rect1;lRet = UTex->LockRect(0, &d3d_rect1, 0, 0);if (FAILED(lRet)){    return false;}// Copy pixel data to texturebyte *pDest1 = (byte *)d3d_rect1.pBits;int stride1 = d3d_rect1.Pitch; for(int i = 0;i < Height/2;i ++){    memcpy(pDest1 + i * stride1 / 2,plane[1] + i * Width / 2, Width / 2);}UTex->UnlockRect(0);D3DLOCKED_RECT d3d_rect2;lRet =  VTex->LockRect(0, &d3d_rect2, 0, 0);if (FAILED(lRet)){    return false;}// Copy pixel data to texturebyte *pDest2 = (byte *)d3d_rect2.pBits;int stride2 = d3d_rect2.Pitch; for(int i = 0;i < Height/2;i ++){    memcpy(pDest2 + i * stride2 / 2,plane[2] + i * Width / 2, Width / 2);}VTex->UnlockRect(0);

Rendering:

Device->BeginScene();Device->SetPixelShader(MultiTexPS);Device->SetFVF(MultiTexVertex::FVF);Device->SetStreamSource(0, QuadVB, 0, sizeof(MultiTexVertex));// Y texDevice->SetTexture(     YTexDesc.RegisterIndex, YTex);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);// U texDevice->SetTexture(     UTexDesc.RegisterIndex, UTex);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);// string texDevice->SetTexture(     VTexDesc.RegisterIndex, VTex);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER);Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);Device->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2);Device->EndScene();Device->Present(0, 0, 0, 0);

 

D3D Texture rendering YUV420P mainly involves the above content. Complete Engineering Code: http://download.csdn.net/download/qq_33892166/9702415

The pixel coloring machine image written in the HLSL code in this article is somewhat yellow. If you know how to optimize it, please advise.