Target detection: Selective search strategy (C + +/Python)

Guide: Through this tutorial, we will thoroughly understand an important concept: the common method of target detection "selective Search". OpenCV code using C + + or Python will also be given at the end of this article. target detection vs target recognition

Target recognition solves what is the problem, and the target detection solves the problem.

The core of all target detection algorithms is a target recognition algorithm. Suppose we train a target recognition model that can identify dogs from the region (pathes), a model that tells us if there are any dogs in the patch, it does not give the dog coordinates.

In order to locate the target, we have to select the pathes of the image, and then apply the target recognition algorithm to the image blocks. The position of the target is given by the position of the image sub region with high class probability returned by the target recognition algorithm.

The most direct way to generate a smaller child region (Pathes) is called a sliding window method. However, the sliding window method has several limitations. These limitations are overcome by a class of algorithms called "Zone Recommendation" algorithms. Selective search is the most popular area-recommended algorithm. sliding window algorithm/sliding Windows algorithm

In the sliding window method, we slide a box or window over the image to select an area and use the target recognition model to classify each image block that the window covers. This is an exhaustive search of the entire image of the object. Not only do we need to search all possible locations in the image, but we also have to search on different scales. This is because the object recognition model is usually trained on a specific scale (or range). This will classify thousands of image blocks.
　　
The problem does not end there. The sliding window method is good for fixed aspect ratio objects, such as human faces or pedestrians. The image is a two-dimensional projection of three-dimensional objects, and the object characteristics, such as the aspect ratio and shape, vary significantly according to the angle of the image taken. The way to slide a window becomes very expensive because it needs to search for multiple aspect ratios. region-recommended algorithms/region Proposal algorithms

The problems we have discussed at the moment can be solved by the regional recommendation algorithm. These methods use the image as an input and output bounding box, which corresponds to all the child regions of the image that are most likely to be objects. These regional recommendations may be noisy, overlapping, and may not contain objects entirely, but in these regional recommendations, there will be a suggestion that is very close to the actual object in the image. We can then use the object recognition model to categorize these proposals. Regional recommendations with high probability scores are the location of the objects.

The regional proposed algorithm uses the segmentation method to identify the foreground object in the image. In the segmentation we think that the adjacent areas are similar to each other, based on a number of criteria, such as color, texture, and other sliding window methods, we are looking for all the pixel position and at all scale objects, the region algorithm works by grouping pixels to a smaller number of segments. Therefore, the proposed final quantity is many times less than the sliding window approach. This reduces the number of image blocks that we have to classify. These generated regional recommendations have different scales and aspect ratios.
　　
At present, several methods of regional suggestion are put forward, such as

1. objectness
2. Constrained parametric min-cuts for Automatic Object segmentation
3. Category Independent Object proposals
4. Randomized Prim
5. Selective Search

In all of these recommended methods, selective search is the most common, because it is fast and has a high recall rate. selective search for target recognition

What is a selective search.

Selective search is a regional recommendation algorithm for target detection. Its design speed is fast, the recall rate is high. It calculates hierarchical groupings of similar areas based on the compatibility of color, texture, size, and shape.
　　
Selective search begins the image segmentation of pixels based on the use of graphs by Felzenszwalb and huttenlocher segmentation methods. The output of the algorithm is shown below. The image on the right contains a segmented area represented by a solid color.
　　　　　　　　　

Can we use the segmented section in this image as a regional recommendation? The answer is no, there are two reasons why we can not do this:

1. Most of the actual objects in the original image contain 2 or more segmented sections.
2. In this way, it is not possible to make recommendations for objects that are obscured, such as a cup-covered plate or a cup filled with coffee.

If we try to solve the first problem by further merging similar adjacent areas, we'll get a segmented area that covers two objects. The perfect division is not our goal. We are only trying to anticipate the recommendations of many regions, some of which should have a high overlap with actual objects. Selective search uses the Oversegments felzenszwalb Huttenlocher method as the initial seed. The segmented image looks like this:

　　　　　　　　　　　　　　　　

Select the search algorithm to use these oversegments as the initial input and perform the following steps: Add all bounding boxes for the segment section to the regional suggestion list. Group adjacent segments based on similarity

Go to step 1

In each iteration, a larger segment is generated and added to the regional suggestions list. Therefore, we use the bottom-up approach to create regional recommendations from smaller sections to larger ones. This is what we call "hierarchies" based on computed partitioning using Huttenlocher oversegments.
　　　　　　　　　　　　　　　　

The image shows the initial, intermediate, and final steps of the layered segmentation process. Among the various similarities referenced in the text:
http://www.learnopencv.com/selective-search-for-object-detection-cpp-python/ Results

The OPENCV implements a descending sequential array of object names that are recommended for selective search zones. For clarity, we share results with top 200-250 box paintings in the image. The general recommendation in 1000-1200 is good enough to make all the correct areas.
　　
　　　　　　　　　　　　　　 Selective Search Code

Let's look at how to implement a segmentation based on selective search in OpenCV.

Selective search:c++

#include "opencv2/ximgproc/segmentation.hpp" #include "opencv2/highgui.hpp" #include "opencv2/core.hpp" #include "
Opencv2/imgproc.hpp "#include <iostream> #include <ctime> using namespace CV;

using namespace Cv::ximgproc::segmentation; static void Help () {std::cout << std::endl << Usage: "<< std::endl <<"./ssearch INP  Ut_image (f|q) "<< Std::endl <<" F=fast, q=quality "<< std::endl <<" use L to display less
Rects, M to display more rects, q to Quit "<< Std::endl; int main (int argc, char** argv) {//If image path and f/q is not passed as command//line arguments, quit a
        D Display Help message if (ARGC < 3) {help ();
    return-1;
    }//speed-up using Multithreads setuseoptimized (true);

    Setnumthreads (4);
    Read image Mat im = Imread (argv[1]);
    Resize image int newheight = 200;
    int newwidth = im.cols*newheight/im.rows; ResiZe (IM, IM, Size (newwidth, newheight)); Create selective Search segmentation Object using default parameters ptr<selectivesearchsegmentation> SS = CR
    Eateselectivesearchsegmentation ();

    Set input image on which we'll run segmentation ss->setbaseimage (IM); Switch to fast but-recall selective Search method if (argv[2][0] = = ' F ') {Ss->switchtoselectivesear
    Chfast (); //Switch to high recall but slow selective Search method else if (argv[2][0] = = ' Q ') {Ss->switchtos
    Electivesearchquality ();
        }//If argument is neither F nor Q print help message else {help ();
    Return-2;
    //Run Selective search segmentation on input image std::vector<rect> rects;
    Ss->process (rects);

    Std::cout << "Total number of Region proposals:" << rects.size () << Std::endl;
    Number of region proposals to show int numshowrects = 100; IncrementTo increase/decrease total number//The reason proposals to be shown int increment = 50;

        while (1) {//Create a copy of original image Mat Imout = Im.clone (); Itereate the region proposals for (int i = 0; i < rects.size (); i++) {if (I < Numsho
            wrects) {Rectangle (imout, rects[i], Scalar (0, 255, 0));
            } else {break;

        }//Show output imshow ("Output", imout);

        Record key Press int k = Waitkey ();
            M is pressed if (k = = 109) {//Increase total number of rectangles to show by increment
        Numshowrects + = increment; }//L is pressed else if (k = 108 && numshowrects > Increment) {//Decrease Tota
        L number of rectangles to show by increment numshowrects-= increment;
        }//q is pressedelse if (k = = 113) {break;
} return 0; }

Selective Search:python

#!/usr/bin/env python ' Usage:./ssearch.py input_image (f|q) f=fast, q=quality use "L" to display less rects, '
M ' To display more rects, "Q" to quit. ' ' Import sys import CV2 if __name__ = = ' __main__ ': # If image path and f/q is not passed as command # line ARG Uments, quit and display Help message if Len (SYS.ARGV) < 3:print (__doc__) sys.exit (1) # speed
    -up using Multithreads cv2.setuseoptimized (True);

    Cv2.setnumthreads (4); # Read Image im = Cv2.imread (sys.argv[1]) # Resize Image newheight = newwidth = Int (im.shape[1]*200/im . shape[0]) im = Cv2.resize (IM, (newwidth, Newheight)) # Create selective Search segmentation Object using Def Ault Parameters SS = Cv2.ximgproc.segmentation.createSelectiveSearchSegmentation () # Set input image on which we Would run segmentation ss.setbaseimage (IM) # Switch to fast but-recall selective Search method if (sys.arg
        V[2] = = ' F '):Ss.switchtoselectivesearchfast () # Switch to high recall but slow selective Search method elif (sys.argv[2] = = ' Q
        '): Ss.switchtoselectivesearchquality () # If argument is neither F nor Q print help message else: Print (__doc__) sys.exit (1) # Run selective search segmentation on input Image rects = ss.process () pr Int (' Total number of Region proposals: {} '. Format (len (rects)) # Number of Region proposals to show numshowrects = M increment to increase/decrease total number # of reason proposals to be shown increment = Whil E True: # Create a copy of original image imout = Im.copy () # Itereate over all Region propos
            ALS for I, rect in enumerate (rects): # Draw rectangle for region proposal till  if (I < numshowrects): X, Y, W, h = rect cv2.rectangle (imout, (x, y), (x+w, Y+h), (0, 255, 0), 1, Cv2. Line_aaElse:break # Show Output cv2.imshow (' Output ', imout) # record key Press k = Cv2.waitkey (0) & 0xFF # M is pressed if k = = 109: # Increase Total num ber of rectangles to show by increment numshowrects + = increment # L is pressed elif k = = 108 and numshowrects > Increment: # Decrease total number of rectangles to show by increment Numsho Wrects-= increment # Q is pressed elif k = = 113:break # Close image Show Window Cv2. Destroyallwindows ()

BUG: In the code above, a bug appears in the Python bindings for the selective search. So Python code uses OPENCV 3.3.0 instead of OpenCV 3.2.0 work. If you don't want to compile OpenCV 3.3.0, build OpenCV 3.2.0 the folders you compile before you can fix the bug. If you look at the GitHub, it's just a small change. You have to change in file line # 239
Opencv_contrib-3.2.0/modules/ximgproc/include/opencv2/ximgproc/segmentation.hpp

From
cv_wrap virtual void process (std::vector<rect>& rects) = 0;
To
cv_wrap virtual void process (Cv_out std::vector<rect>& rects) = 0;

Now you re compiling OpenCV 3.2.0. If you have a build folder that was compiled earlier in the OPENCV, running the Make command compiles the module.