OpenGL and OpenCV for augmented reality

Long time no blog, recently in the study of computer vision related knowledge, so wrote a small AR demo.

The program realizes the recognition and localization of marker by OPENCV, and then overlays the virtual object into the camera image through OpenGL to realize augmented reality. First take a look at the marker we use:

This is one of the many marker, they are surrounded by a circle of black border, the border is encoded information, white for 1, black for 0. Use each line as a single word, then each word has 5bits. Among them, 1, 3, 5 bits is the check digit, 2, 4 bits is the information bit. That is, the information bit for the entire marker is only 10bits, so it can represent a maximum of 1024 numbers (0~1023). This encoding is actually a variant of Hamming code, the only difference is that the first is corresponding to the first of the Hamming code of the inverse (such as Hamming code is 00000, then the code in marker is 10000). This is done to prevent a row from being completely black, thereby increasing the recognition rate. There is another advantage of Hamming code-there is no rotational symmetry, so the program can determine the direction of the marker through the Hamming codes, so the information decoded from the marker is unique.

I. Detection and identification of marker

We first implement a class that detects the marker in the image, decodes the information, and calculates the coordinate position of the marker relative to the camera.

The detection section is relatively simple. First, the input image is transformed into gray scale, then the gray image is adaptively binary. Adaptive binary is used because it can better adapt to the change of illumination. One thing to note, however, is that many friends use adaptive binary representation to obtain results that are much like the result of edge detection, due to the fact that the adaptive window is too small. When using adaptive binary, the size of the window should be greater than the size of the binary target, otherwise the resulting threshold value is not adaptable. After the adaptive binary, in order to eliminate noise or small pieces, can be morphological open operation. The above can be obtained by the following images (of which two of the value of the results have been reversed color processing, convenient for later contour extraction).

After you get the two value image, you can use the findcontours in OpenCV to extract the contour. There are a number of contours in a binary image, some of which are small, and we exclude these too small outlines with a threshold value. Once you have ruled out a small outline, you can make a polygonal approximation of the contour. Since our marker is a square, the polygon approximation results should meet the following conditions:

1. Only 4 vertices

2, must be convex polygon

3, each side of the length can not be too small

With these conditions, we can exclude most of the contours and find the most likely part of the marker. After finding such a candidate contour, we save its polygon four vertices and make the appropriate adjustments so that all vertices are sorted counterclockwise. The code is as follows:

[CPP]  View Plain  copy   Void markerrecognizer::markerdetect (mat& img_gray, vector <marker>& possible_markers, int min_size, int min_side_length)    {        Mat img_bin;          int  thresh_size =  (MIN_SIZE/4) *2 + 1;       adaptivethreshold (img _gray, img_bin, 255, adaptive_thresh_gaussian_c, thresh_binary_inv, thresh_size, &NBSP;THRESH_SIZE/3);       //threshold (img_gray, img_bin, 125,  255, thresh_binary_inv| Thresh_otsu);       morphologyex (Img_bin, img_bin, morph_open, mat ());  //use open operator to eliminate small patch       &NBSP;&NBSP;&NBSP;&NBSP;VECTOR&LT;VECTOR&LT;POINT&GT;&GT;&NBSP;ALL_CONTOURS;&NBSP;&NBsp     vector<vector<Point>> contours;        Findcontours (img_bin, all_contours, cv_retr_list, cv_chain_approx_none);           for  (Int i = 0; i < all_contours.size ();  ++i)        {           if   (All_contours[i].size ()  > min_size)             {               contours.push_back ( All_contours[i]);           }        }          vector<Point> approx_poly;        for  (Int i = 0; i < contours.size ();  ++i)     &Nbsp;   {           double eps =  contours[i].size () *approx_poly_eps;            APPROXPOLYDP (contours[i], approx_poly, eps, true);               if  (Approx_poly.size ()  != 4)                 continue;               if  (!iscontourconvex (approx_poly))                 continue;               //ensure that the distance between consecutive points is  large enough           float min_side =  FLT_MAX;           for  (int j = 0; j < 4; ++j )            {                point side = approx_poly[j] - approx_poly[(j+1)%4 ];               min_side = min (Min_size, side.dot (side));           }           if  (Min_side < min_side_length*min_side_ Length)                continue;