Multi-level processing of pictures in WebGL (FrameBuffer)

In the process of using WEBGL, we often want to texture and post it on the surface to display

For more accurate edge detection, we have to use grayscale shader, fuzzy shader, edge shader for processing, and each processing object is the last processing of texture, This will overwrite and save the processed results.

In the many WebGL libraries, there is a direct option redertotarget to render the shader processed texture and overwrite the original texture, how does it complete this step?

This leads to the protagonist of this article--framebuffer

What is Framebuffer?

FBO (Frame Buffer object) is an extension that is recommended for rendering data to texture objects .

Framebuffer is like a WebGL display container, Usually we use gl.drawarrays or gl.drawelements to draw the object in the default window, and when we specify a framebuffer as the current window, the two methods are used to draw, then the object is drawn in the specified framebuffer.

Use of framebuffer

Internalformat, int x, int y, Sizei width,
Sizei height, int border);
target:texture_2d, Texture_

Creation of FBO:

// Create a framebuffer var fb = gl.createframebuffer (); // bind FB as the current window    Gl.bindframebuffer (GL. FRAMEBUFFER,FB);

In this way, we create a new buffer that can be drawn, and it is not displayed.

But will that be all right? The idea is to render the texture after the shader rendering and hand it over to the next shader, when the method is introduced framebuffertexture2d

Reference from "OpenGL ES Reference Pages about Framebuffertexture2d":

To render directly-a texture image, a specified image from a texture object can be attached as one of the logical buf Fers of the currently bound Framebuffer object by calling the command

In order to render directly into a texture picture, a picture specified in a texture object can be bound to the FBO of the current use in the logical cache using the following method

void Framebuffertexture2d (enum target, enum attachment, enum Textarget, uint texture, int level);

Target

• FRAMEBUFFER

Attachment

if attachment is color_attachment0and then image must has a colorrenderable internal format. (color)

if attachment is depth_attachmentand then image must has a depthrenderable internal format. (Depth)

if attachment is stencil_attachmentand then image must has a stencilrenderable internal format. (template)

Textarget:

• texture_2d (two-dimensional TEXTURE)

• texture_cube_map_positive_x (three-dimensional +x TEXTURE)

• texture_cube_map_positive_y (three-dimensional +y TEXTURE)

• texture_cube_map_positive_z (three-dimensional +z TEXTURE)

• texture_cube_map_negative_x (three-dimensional-x TEXTURE)

• texture_cube_map_negative_y (three-dimensional-y TEXTURE)

• texture_cube_map_negative_z (three-dimensional-z TEXTURE)

Texture

　　Texture Object

Level

Specifies the mipmap level of the texture image to being attached to the framebuffer and must is 0.

We use this method to bind (this article only describes color bindings, try to be similar to templates, but there are differences, not discussed here)

//Create a Texture objectvarTexture =gl.createtexture ();//use the following settings to create the texture, so that the texture setting allows us to manipulate images of any sizeGl.bindtexture (GL. texture_2d, TEXTURE); Gl.texparameteri (GL. TEXTURE_2D, GL. texture_wrap_s, GL. Clamp_to_edge); Gl.texparameteri (GL. TEXTURE_2D, GL. texture_wrap_t, GL. Clamp_to_edge); Gl.texparameteri (GL. TEXTURE_2D, GL. Texture_min_filter, GL. NEAREST); Gl.texparameteri (GL. TEXTURE_2D, GL. Texture_mag_filter, GL. NEAREST);
varFB =Gl.createframebuffer (); Gl.bindframebuffer (GL. FRAMEBUFFER,FB);//Use this method to bind a texture color value to FBOGl.framebuffertexture2d (GL. FRAMEBUFFER, GL. COLOR_ATTACHMENT0, GL. texture_2d, TEXTURE, 0);

After binding, when we execute the Gl.drawarrays or Gl.drawelements method, it will render directly to the currently bound FBO, while FBO is bound to the texture color, so the color is also rendered to texture when drawn.

In this way, we can use two FBO to process the queue:

Originalimage-Texture1

Texture1---Texture2

Texture2--Blur-Texture1

Texture1--edge-to-texture2

The following is a concrete implementation process

varFBOs =[], Textures= [];  for(vari = 0; I < 2; i++){    varTexture =gl.createtexture (); Gl.bindtexture (GL.    texture_2d, TEXTURE); Gl.texparameteri (GL. TEXTURE_2D, GL. texture_wrap_s, GL.    Clamp_to_edge); Gl.texparameteri (GL. TEXTURE_2D, GL. texture_wrap_t, GL.    Clamp_to_edge); Gl.texparameteri (GL. TEXTURE_2D, GL. Texture_min_filter, GL.    NEAREST); Gl.texparameteri (GL. TEXTURE_2D, GL. Texture_mag_filter, GL.    NEAREST); varFB =Gl.createframebuffer (); Gl.bindframebuffer (GL.    FRAMEBUFFER,FB); Gl.framebuffertexture2d (GL. FRAMEBUFFER, GL. COLOR_ATTACHMENT0, GL. texture_2d, TEXTURE,0); //store corresponding texture and FBTextures.push (texture); Fbos.push (FB);} Gl.bindtexture (GL. texture_2d, originalimagetexture); for(vari = 0; I < 4; i++){    Switch(i) { Case0:            //set show shader to current shader program            //handle arguments to vs shader and FS shader            //Draw original image into texture which is bound to a FBO             Break;  Case1:            //set gray shader to current shader program            //handle arguments to vs shader and FS shader             Break;  Case2:            //set blur shader to current shader program            //handle arguments to vs shader and FS shader             Break;  Case3:            //Set edge shader to current shader program            //handle arguments to vs shader and FS shader             Break; } gl.bindframebuffer (GL. FRAMEBUFFER, Fbos[i%]); //set the viewport fits the images sizeGl.viewport (0, 0, ImgWidth, imgheight); Gl.drawarrays (...); //or gl.drawelements (...);        //set the rendered texture to current texture for next frambuffer usingGl.bindtexture (GL. texture_2d, texture[i%2]);}

The complete process is:

set FBO1 ---draw--FBO1-- set Texture1set FBO2 ----and draw--FBO2 - Set Texture2set FBO1 ---draw--FBO1 set texture1set FBO2set Texture2 

In this process, FBO1 and texture1 are color-rendered bindings, so the set FBO1 renders directly to the Texture1

When we have finished the entire drawing, to display the processed picture normally, we need to jump from the FBO :

// set FBO to NULL to use default Framebuffer null);

Other uses of framebuffer

Gl.readpixels

Reading pixel color data from framebuffer

Reference from "Webgl_2.0_reference_card"/"OpenGL ES reference Pages about Readpixels":

Pixels in the current Framebuffercan is read back to an Arraybufferview object.

void Readpixels (int x, int y, long width, long height,enum format, enum type, Object pixels)

X, y

Specify the window coordinates of the first pixel that's read from the frame buffer. This was the lower left corner of a rectangular block of pixels.

Width,height

Specify the dimensions of the pixel rectangle. And of one width height correspond to a single pixel.

Format

Specifies the format of the pixel data. The following symbolic values is accepted RGBA  in WebGL

Type

Specifies the data type of the pixel data. Must bes UNSIGNED_BYTE in WebGL

Pixels

returns the pixel data.

In the process of use, we need to create a pixels object to store the data

// using Arraybufferview to store pixels data only, Unit8array are the best because each color data is a byte var New Uint8array (ImageWidth * imageheight * 4); Gl.readpixels (0, 0, ImageWidth, ImageHeight, GL. RGBA, GL. Unsigned_byte, pixels);

In this way, we can get the color data from the whole FBO.

GL. Copyteximage2d GL. Copytexsubimage2d

Both of these functions are used to copy data from FBO to the texture of the current binding.

Copyteximage2d Method:

Reference from "OpenGL ES Reference Pages about copyteximage2d":

Copy pixels into a 2D texture image

void Copyteximage2d (enum target, int level,enum internalformat, int x, int y, Sizei width,sizei height, int border);

Target

texture_2d

texture_cube_map_positive_{x, Y, Z},

texture_cube_map_negative_{x, Y, Z}

Internalformat:

Alpha

Luminance

Luminance_alpha

RGB

Rgba

X, y

Specify the window coordinates of the lower left corner of the rectangular region of pixels to be copied.

Width

Specifies the width of the texture image. Must be 0 or 2 n + 2? Border for some integer n.

Height

Specifies the height of the texture image. Must be 0 or 2 m + 2? Border for some integer m.

Border

Specifies the width of the border. Must be either 0 or 1.

Copytexsubimage2d Method:

Reference from "OpenGL ES Reference Pages about copytexsubimage2d":

Copy a two-dimensional texture subimage

void Copytexsubimage2d (enum target, int level, int xoffset,int yoffset, int x, int y, Sizei width, sizei height);

Target

texture_2d

texture_cube_map_positive_{x, Y, Z},

texture_cube_map_negative_{x, Y, Z}

Level

Specifies the level-of-detail number. Level 0 are the base image level. Level n is the nth mipmap reduction image.

Xoffset:

Specifies a texel offset in the x direction within the texture array.

Yoffset:

Specifies a texel offset in the y direction within the texture array.

X, Y:

Specify the window coordinates of the lower left corner of the rectangular region of pixels to be copied.

Width

Specifies the width of the texture subimage.

Height

Specifies the height of the texture subimage.

The difference between the two methods is that you can see it.

Copytexsubimage2d relative copyteximage2d adds offset to change the copy area

Its final copy area is: [x, Xoffset + width-1] with [y, Yoffset + height-1].

The copyteximage2d is more than the copytexsubimage2d Internelformat parameters to control the type of pixel data replication.

Conclusion:

With a flexible operation on texture, we can make something more interesting, and framebuffer is a very important part of it.

attached:

WebGL-1.0 Reference Card: http://files.cnblogs.com/files/zhiyishou/webgl-reference-card-1_0.pdf

opengl-es-2.0 Reference Card: http://files.cnblogs.com/files/zhiyishou/OpenGL-ES-2_0-Reference-card.pdf

opengl-es-2.0 Reference Manual: https://www.khronos.org/opengles/sdk/docs/man/

The end.

Multi-level processing of pictures in WebGL (FrameBuffer)