Opencv for iOS Study Notes (9)-rendering ar scenario 2 with OpenGL

Original address: opencv for iOS Study Notes (9)-rendering ar scenario 2 with OpenGL

The operation procedure of drawframe is:

1. Clear the scenario.

2. Start the normal projection to draw the background.

3. Draw the last image obtained from the camera in the viewport.

4. Set Perspective Projection based on internal camera parameters.

5. Move the coordinate system of each detected mark to the three-dimensional position of the mark (apply the 4x4 transformation matrix to the OpenGL model matrix ).

6. Present an arbitrary 3D object.

7. display frame cache.

We will call drawframe when frames are ready-when a new camera frame is uploaded to the video memory, and the marking and monitoring steps have been completed.

The following is the drawframe code:

-(Void) drawframe {// start the activated frame cache [m_glview setframebuffer]; // draw the background [self drawbackground]; // draw a 3D object [self drawar] At the detected mark; // render the frame cache bool OK = [m_glview presentframebuffer]; int glercode = glgeterror (); If (! OK | glercode! = Gl_no_error) {STD: cerr <"OpenGL detection error. Error Code:" <glercode <STD: Endl ;}}

It is quite easy for us to draw a background. We enable face-to-face projection and draw a full screen image texture of the current frame. The following is the OpenGL ES1 code:

-(Void) drawbackground {// obtain the glfloat W = ceiling; glfloat H = m_glview.bounds.size.height; // The Four vertex coordinates const glfloat squarevertices [] = {0, 0, w, 0, 0, H, W, h}; // texture vertex static const glfloat texturevertices [] = {1, 0, 1, 1, 0, 0, 0, 1}; // face matrix static const glfloat proj [] = {0,-2.f/ W, 0, 0,-2.f/ H, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1};/* glmatrixmode-specifies which matrix is specified by the current matrix mode Matrix stacks are the targets of the next matrix operation. Optional values: gl_modelview, gl_projection, and gl_texture. description: glmatrixmode sets the current matrix mode: gl_modelview, which applies subsequent matrix operations to the model visual matrix stack. gl_projection: Apply subsequent matrix operations to the projection matrix. gl_texture: Apply subsequent matrix operations to the texture matrix stack. use glloadidentity () together with glloadidentity (): This function is used to reset the currently specified matrix as the unit matrix. After glloadidentity (), we set a perspective for the scenario. Glmatrixmode (gl_modelview) sets the current matrix as a model view matrix, which stores information about objects. */Glmatrixmode (gl_projection); glloadmatrixf (proj); glmatrixmode (gl_modelview); glloadidentity (); random (false); gldisable (random); glable (random); glbindtexture, m_backgroundtextureid); // update the attribute value. glvertexpointer (2, gl_float, 0, squarevertices); glableclientstate (gl_vertex_array); random (2, gl_float, 0, random); glableclientstate (random); glcolor4f (1, 1 ); gldrawarrays (gl_triangle_strip, 0, 4); gldisableclientstate (gl_vertex_array); gldisableclientstate (gradient); gldisable (gl_texture_2d );}

Rendering virtual objects in the scene is clever. First, we need to adapt to the OpenGL projection matrix according to the internal parameters of the camera. Without this step, we will get the wrong Transmission Projection, which will make the virtual object look less natural, as if "floating in the air", no sense of reality! Correct projection is essential for all augmented reality applications.

The following code creates an OpenGL projection matrix based on internal camera parameters:

-(Void) buildprojectionmatrix :( matrix33) cameramatrix: (INT) screen_width: (INT) screen_height: (matrix44 &) projectionmatrix {float near = 0.01; // float far = 100; // float far =; // The Camera Parameter float f_x = cameramatrix. data [0]; // focal length in X axis float f_y = cameramatrix. data [4]; // focal length in y axis (usually the same ?) Float C_x = cameramatrix. data [2]; // camera primary point X float c_y = cameramatrix. data [5]; // camera primary point y projectionmatrix. data [0] =-2.0 * f_x/screen_width; projectionmatrix. data [1] = 0.0; projectionmatrix. data [2] = 0.0; projectionmatrix. data [3] = 0.0; projectionmatrix. data [4] = 0.0; projectionmatrix. data [5] = 2.0 * f_y/screen_height; projectionmatrix. data [6] = 0.0; projectionmatrix. data [7] = 0.0; projectionmatrix. data [8] = 2.0 * C_x/screen_width-1.0; projectionmatrix. data [9] = 2.0 * c_y/screen_height-1.0; projectionmatrix. data [10] =-(far + near)/(far-near); projectionmatrix. data [11] =-1.0; projectionmatrix. data [12] = 0.0; projectionmatrix. data [13] = 0.0; projectionmatrix. data [14] =-2.0 * far * near/(far-near); projectionmatrix. data [15] = 0.0 ;}

After we load this matrix into the OpenGL pipeline, we will draw an object.

Any transformation can be rendered by a 4x4 matrix and loaded into the OpenGL Model View matrix. This step moves the coordinate system to the mark in the world coordinate system.

For example, let's draw a coordinate axis above each tag-it will show the marked direction in space and fill the entire tag with a gradient rectangle

. This visual operation will give us visual feedback as expected.

The following is the implementation of the drawar function:

-(Void) drawar {matrix44 projectionmatrix; // The projection matrix to be output at the width and height of the internal screen of the camera [self buildprojectionmatrix: m_calibration.getintrinsic (): m_framesize.width: m_framesize.height: glmatrixmode (gl_projection); glloadmatrixf (projectionmatrix. data); glmatrixmode (gl_modelview); glloadidentity (); gldepthmask (true); glable (latency); // gldepthfunc (gl_less); // gldepthfunc (gl_greater ); glableclientstate (gl_vertex_array); glableclientstate (gl_normal_array); glpushmatrix (); gllinewidth (3.0f); // three axes float LinEx [] = {0, 0, 1, 0, 0 }; float Liney [] = {,}; float linez [] = {,}; const glfloat squarevertices [] = {-0.5f,-0.5f, 0.5f,-0.5f,-0.5f, 0.5f, 0.5f, 0.5f,}; const glubyte squarecolors [] = {255,255, 0,255, 0,255,255,255, 0, 0, 0, 0,255, 0,255,255 ,}; // traversal tag transformation for (size_t transformationindex = 0; transformationindex <m_transformations.size (); transformationindex ++) {const Transformation & Transformation = m_transformations [transformationindex]; matrix44 glmatrix = transformation. getmat44 (); glloadmatrixf (reinterpret_cast <const glfloat *> (& glmatrix. data [0]); // draw glvertexpointer (2, gl_float, 0, squarevertices); glableclientstate (gl_vertex_array); glcolorpointer (4, vertex, 0, squarecolors ); glableclientstate (gl_color_array); gldrawarrays (values, 0, 4); gldisableclientstate (gl_color_array); float scale = 0.5; glscalef (scale, scale, scale); gltranslatef (0, 0, 0.1f); glcolor4f (1.0f, 0.0f, 0.0f, 1.0f); glvertexpointer (3, gl_float, 0, LinEx); gldrawarrays (gl_lines, 0, 2); glcolor4f (0.0f, 1.0f, 0.0f, 1.0f); glvertexpointer (3, gl_float, 0, Liney); gldrawarrays (gl_lines, 0, 2); glcolor4f (0.0f, 0.0f, 1.0f, 1.0f); glvertexpointer, gl_float, 0, linez); gldrawarrays (gl_lines, 0, 2);} glpopmatrix (); gldisableclientstate (gl_vertex_array );}

At the end of an example, we will demonstrate our results and summarize them.

Opencv for iOS Study Notes (10)-mark detection Summary