Cesium principle: 6 render Module (4:FBO)

Cesium not only provides FBO, which is the Framebuffer class, but the entire rendering process is performed in FBO. FBO, Chinese is the frame buffer, usually is a high-level usage, but in fact, if you understand its basic principles, it is very simple to use, the key is to understand. For example, you build a building, the foundation did not play well, cover the first floor, can also, cover the second floor, a little down, cover the third floor, collapsed. You would think the third floor (FBO) is too difficult, in fact the root cause is still on the foundation.

The frame caches managed by the Windows system have their own cache objects (color, depth, and template), which are created before the window is created, and we create the framebuffer ourselves, which are required to be built manually. Or, by default, you know the concept of FBO and the way it is used in WebGL, and on that basis we look at the encapsulation of FBO in cesium. First look at the main attributes in FBO:

 
 
function Framebuffer(options) {
    var gl = options.context._gl;
    var maximumColorAttachments = ContextLimits.maximumColorAttachments;

    this._gl = gl;
    this._framebuffer = gl.createFramebuffer();

    this._colorTextures = [];
    this._colorRenderbuffers = [];
    this._activeColorAttachments = [];

    this._depthTexture = undefined;
    this._depthRenderbuffer = undefined;
    
    this._stencilRenderbuffer = undefined;
    
    this._depthStencilTexture = undefined;
    this._depthStencilRenderbuffer = undefined;
}

       through properties, Cesium's FBO mainly supports two ways of rendering to texture (RTT) and render to render buffer (RBO). And the two methods are basically the same in use, two choose one, of course, there can be more than one color texture (cache), as long as no more than the maximumcolorattachments limit. The frame cache attachment is also provided to save the rendering result, which provides the ability to write multiple caches simultaneously (MRT), enabling some multi-screen and split-screen effects. Personally think that renderbuffer performance better, as far as possible to reduce the consumption of data consumption, in the ability to support the two are similar, are off-screen rendering. But textures can be used independently, and RBO data must be linked to a frame cache object to make sense. Let's take a look at how it was created:

globeDepth.framebuffer = new Framebuffer({
    context : context,
    colorTextures : [globeDepth._colorTexture],
    depthStencilTexture : globeDepth._depthStencilTexture,
    destroyAttachments : false
});

this._fb = new Framebuffer({
    context : context,
    colorTextures : [new Texture({
        context : context,
        width : width,
        height : height
    })],
    depthStencilRenderbuffer : new Renderbuffer({
        context : context,
        format : RenderbufferFormat.DEPTH_STENCIL
    })
});

Personally, Renderbuffer compared to rendertexture way better, but the former in the use of a lot of restrictions, use is not convenient, The key is that there are some interfaces are WebGL2.0 standard, poor compatibility, such as Glblitframebuffer, so in many cases, if we want to read the cache object, generally use texture way. Let's take a look at the internal construction process when we new framebuffer:

 
// Bind FBO
Framebuffer.prototype._bind = function() {
    var gl = this._gl;
    gl.bindFramebuffer(gl.FRAMEBUFFER, this._framebuffer);
};
// release FBO
Framebuffer.prototype._unBind = function() {
    var gl = this._gl;
    gl.bindFramebuffer(gl.FRAMEBUFFER, null);
};

// Bind color texture, specify frame buffer attachment attachment
function attachTexture(framebuffer, attachment, texture) {
    var gl = framebuffer._gl;
    gl.framebufferTexture2D(gl.FRAMEBUFFER, attachment, texture._target, texture._texture, 0);
}
// Bind the rendering cache object, specify the frame cache attachment attachment
function attachRenderbuffer(framebuffer, attachment, renderbuffer) {
    var gl = framebuffer._gl;
    gl.framebufferRenderbuffer(gl.FRAMEBUFFER, attachment, gl.RENDERBUFFER, renderbuffer._getRenderbuffer());
}

function Framebuffer(options) {
    this._bind();
    if (defined(options.colorTextures)) {
        // Check whether the number of color textures exceeds the upper limit
        length = this._colorTextures.length = this._activeColorAttachments.length = textures.length;
        if (length> maximumColorAttachments) {
            throw new DeveloperError(‘The number of color attachments exceeds the number supported.’);
        }

        // Bind the color texture in turn
        for (i = 0; i <length; ++i) {
            texture = textures[i];

            attachmentEnum = this._gl.COLOR_ATTACHMENT0 + i;
            attachTexture(this, attachmentEnum, texture);
            this._activeColorAttachments[i] = attachmentEnum;
            this._colorTextures[i] = texture;
        }
    }
    
    // Similarly, bind rendering cache, depth, template, etc. in sequence
    
    this._unBind();
}

Encapsulates the entire FBO creation process, the user needs simply a few words, cesium is good to complete the encapsulation process. OVER,FBO's usage is over, it's that simple. Here are two cesium in the use of FBO, one is cesium will eventually paste FBO to the screen, one is the implementation of pick.

FBO Patch Screen

Some browsers, perhaps because of the graphics card compatibility issues, such as you use a card, will not support the depth of texture, cesium to do some special considerations. The following logic is an approximate flow in cases where deep textures are supported. First, the FBO is created in globedepth during initialization:

 
function createFramebuffers(globeDepth, context, width, height) {
     // Create a color texture the same size as the current window in GlobeDepth
     globeDepth.framebuffer = new Framebuffer({
         context: context,
         colorTextures: [globeDepth._colorTexture],
         depthStencilTexture: globeDepth._depthStencilTexture,
         destroyAttachments: false
     });
}

And our rendering process is roughly the following:

 
function render(scene, time) {
     // Clear FBO
     var passState = scene._passState;
     passState.framebuffer = undefined;
     // Update passState.framebuffer and render the FBO
     updateAndExecuteCommands(scene, passState, defaultValue(scene.backgroundColor, Color.BLACK));
     // Process FBO and render to the screen
     resolveFramebuffers(scene, passState);
}

First we first understand how to render to the FBO process (updateandexecutecommands), the approximate logic is to choose the appropriate framebuffer, and then the Drawcommand render to the FBO, the key code is as follows:

 
// Called in updateAndExecuteCommands to update passState.framebuffer
function updateAndClearFramebuffers(scene, passState, clearColor, picking) {
    if (environmentState.isSunVisible && scene.sunBloom && !useWebVR) {
        passState.framebuffer = scene._sunPostProcess.update(passState);
    } else if (useGlobeDepthFramebuffer) {
        passState.framebuffer = scene._globeDepth.framebuffer;
    } else if (useFXAA) {
        passState.framebuffer = scene._fxaa.getColorFramebuffer();
    }
}

// Called in updateAndExecuteCommands to start rendering all DrawCommands
function executeCommandsInViewport(firstViewport, scene, passState, backgroundColor, picking) {
    executeCommands(scene, passState);
}
// DrawCommand actually calls the Context.draw method, the next article will introduce DrawCommand in detail
DrawCommand.prototype.execute = function(context, passState) {
    context.draw(this, passState);
};

Context.prototype.draw = function(drawCommand, passState) {
    passState = defaultValue(passState, this._defaultPassState);
    // Get the corresponding FBO, give priority to off-screen rendering
    var framebuffer = defaultValue(drawCommand._framebuffer, passState.framebuffer);

    beginDraw(this, framebuffer, drawCommand, passState);
    continueDraw(this, drawCommand);
};

Visible, framebuffer priority choice _globedepth, followed by _fxaa, and at this time Context.prototype.draw, must be off-screen rendering, that is, rendering to FBO. In addition, the objects we render are encapsulated in a Drawcommand class, such as previous terrain slices, model data, or geometry data, and eventually create a drawcommand to complete the final rendering.

Then is the final step, so Ching Lloyd, we took so much trouble, finally came to the final step. Remember the globedepth._colortexture we created at the time of initialization, bound in globedepth, and the previous FBO process was the rendering of this texture, and now, what we're going to do is say the texture is rendered to FBO in Fxaa, Then FXAA renders it to the screen:

 
function resolveFramebuffers(scene, passState) {
     if (useFXAA) {
         if (!useOIT && useGlobeDepthFramebuffer) {
             // Bind to FBO in FXAA
             passState.framebuffer = scene._fxaa.getColorFramebuffer();
             // Render the _colorTexture of globeDepth to fxaa
             scene._globeDepth.executeCopyColor(context, passState);
         }

         // The framebuffer is empty, that is, rendered to the screen
         passState.framebuffer = environmentState.originalFramebuffer;
         // Render fxaa._texture to the screen
         scene._fxaa.execute(context, passState);
     }
}

In the render-to-screen, FXAA (Fast approximate anti-aliasing) has an anti-aliasing effect in the shader, which is equivalent to a beauty effect on the Earth, which ultimately renders the frame. corresponding to the VEC3 Fxaapixelshader (vec2 Pos, sampler2d Tex, Vec2 Rcpframe) method, the GPU implements the fan aliasing effect.

Pickframebuffer

With the above process, we should have a clear understanding of the use of FBO, below we see if the FBO implementation of the pickup function, corresponding to the Pickframebuffer class. In fact, the idea of picking is very simple, is to have an "id" texture, each rendered object is given a unique ID and the ID to Rgba, when rendered to the "id texture", the rendering is the ID color. When the user clicks to pick up each figure, it looks for the color value in the corresponding ID texture and goes to the ID, and the corresponding figures are found based on the ID. In this process, we can implement the ID texture drawing through FBO and shader, and read the FBO color texture value two technical points. First look at how the ID texture is implemented:

// Construct a PickID object, including the Object, Key and Color
function PickId(pickObjects, key, color) {
     this._pickObjects = pickObjects;
     this.key = key;
     this.color = color;
}

// Provide a method to build PickID
// Ensure that each Ojbect ID is unique
// Convert the ID to the corresponding Color through the Color.fromRgba(key) method
Context.prototype.createPickId = function(object) {
     ++this._nextPickColor[0];
     var key = this._nextPickColor[0];

     this._pickObjects[key] = object;
     return new PickId(this._pickObjects, key, Color.fromRgba(key));
}; 

In this way, updating the figure in rendering is the color (color->pickcolor), which is simple in the GLSL code. While a click event triggers the scene's Pick event:

Scene.prototype.pick = function(windowPosition) {
    var passState = this._pickFramebuffer.begin(scratchRectangle);    
    
    updateAndExecuteCommands(this, passState, scratchColorZero, true);
    resolveFramebuffers(this, passState);

    var object = this._pickFramebuffer.end(scratchRectangle);
}

At this point, the Screenspacerectangle is updated, only the relevant areas of the click are rendered, that is, only the local area is updated, and the FBO in the Pickframebuffer are returned, so the rendering results are saved in the Pickframebuffer frame buffer. Complete the ID texture. Finally, the color value of the corresponding texture is read in PickFramebuffer.prototype.end, and the corresponding object is found to complete the whole picking process. Here is the procedure for getting the color ID corresponding to object:

PickFramebuffer.prototype.end = function(screenSpaceRectangle) {
    var width = defaultValue(screenSpaceRectangle.width, 1.0);
    var height = defaultValue(screenSpaceRectangle.height, 1.0);

    var context = this._context;
    // Get the color value of the click area, RGBA type
    var pixels = context.readPixels({
        x: screenSpaceRectangle.x,
        y: screenSpaceRectangle.y,
        width: width,
        height: height,
        framebuffer: this._fb
    });
    
    var colorScratch = new Color();
    // RGBA is converted to 4 byte arrays, corresponding to a float color component between 0 and 1
    colorScratch.red = Color.byteToFloat(pixels[0]);
    colorScratch.green = Color.byteToFloat(pixels[1]);
    colorScratch.blue = Color.byteToFloat(pixels[2]);
    colorScratch.alpha = Color.byteToFloat(pixels[3]);
    // Get the corresponding Object through the color value and return
    var object = context.getObjectByPickColor(colorScratch);
    if (defined(object)) {
        return object;
    }
    // No Object is selected
    return undefined;
};

Context.prototype.getObjectByPickColor = function(pickColor) {
    // The color value is converted to 4 bytes, which is converted to an int
    // It feels like a circle here
    return this._pickObjects[pickColor.toRgba()];
}; 

Summarize

FBO use simple, powerful, the reason is not easy to understand, but also in the practical application of flexible use, a lot of practical problems can be solved through FBO technology, to achieve the screen processing (in the invisible case, through the shader programmable pipeline, through the coding to achieve efficient and flexible solution), such as FXAA, or ID texture, which is the FBO of the powerful. It can be said that with FBO, we can render any attribute information in our custom format to render buffer objects or textures, and interpret these attributes according to this specification, so that many advanced applications can be extended. and FBO support MRT capability, realize the hardware based on GPU, parallel, through the visualization technology of data processing ability, opened a new window, ushered in a world.

Cesium principle: 6 render Module (4:FBO)