Extracting horizontal and vertical lines based on morphological operations

Objective:

Combined with a custom kernel, two very common morphological operators (e.g., expansion and erosion) are applied to extract horizontal and vertical lines. The following OPENCV functions will be used:

• Cv::erode
• CV::d ilate
• Cv::getstructuringelement

The next example is to extract the notes from the score (the separation of the notes and the score lines in the staff).

Theory

morphology Operations

Morphology is a set of image processing operations based on predefined structuring elements(also known as nuclei). The value of each pixel in the output image is determined based on the comparison between the center pixel and the value of the neighboring pixel in the input image. By selecting the size and shape of the kernel, you can construct a morphological operation that is sensitive to the specific shape of the input image.
The two most basic morphological operations are swelling and corrosion. Bloat adds pixels to the bounds of an object in the image, whereas corrosion is the opposite. The amount of pixels added or removed, depending on the size and shape of the structure element used to process the image. In general, the rules for these two operations are as follows:

• Swell (dilation): The value of the output pixel is the maximum value of all pixels on the size and shape of the structure element. For example, in a binary image, if any one pixel of the input image in the kernel range is set to the value 1, the corresponding pixel of the output image will be set to 1. The latter applies to any type of image (such as grayscale, BGR, etc.).
  

• 
  Expansion of binary images (dilation on a binary image)
  Expansion of grayscale images (dilation on a grayscale image)

• erosion: In contrast to the application of corrosion operations, the value of the output pixel is the minimum value of all pixels on the size and shape of the structure element. is an example:

two-value image corrosion (erosion on a Binary image)

grayscale image corrosion (erosion on a grayscale image)

Structuring Elements (structural element)

as mentioned earlier, the structural elements used to probe the input image are the most important part of any morphological operation.

A structure element is a matrix that contains only 0 and 1, and can have any arbitrary shape and size. Typically, an element with a value of 1 defines an adjacent neighborhood, much less than the processed image. The central pixel of the structure element, called the origin, identifies the pixel of interest (the pixel being processed). For example, here is a diamond structure element with 7x7 elements.

The origin of a diamond-shaped structure element and structure (a diamond-shaped structuring element and its origin)

A structural element can have many common shapes (such as lines, diamonds, discs, periodic lines, and circles) and sizes. You will typically select a structure element that is consistent with the size shape and the object to be processed/extracted. For example, look for lines in an image, create a linear structure element, and you'll see later.

Code

/** * @file morphology_3 (extract_lines). cpp * @brief use morphology transformations for extracting horizontal and vertical Lines Sample code * @author OpenCV Team */#include <iostream> #include <opencv2/opencv.hpp>using namespace std ; using namespace Cv;int main (int, char** argv) {//!    [Load_image]//load the image Mat src = imread (argv[1]);    Check if image is loaded fine if (!src.data) cerr << "Problem loading image!!!" << Endl; Show Source image imshow ("src", src);//! [load_image]//!    [Gray]//Transform source image to Gray if it's not Mat gray;    if (src.channels () = = 3) {Cvtcolor (src, Gray, Cv_bgr2gray);    } else {gray = src; }//Show Gray image Imshow ("Gray", gray);//! [gray]//!    [bin]//Apply adaptivethreshold at the bitwise_not of Gray, notice the ~ symbol Mat BW;    Adaptivethreshold (~gray, BW, 255, Cv_adaptive_thresh_mean_c, Thresh_binary, 15,-2); Show Binary Image IMSHow ("binary", BW);//! [bin]//!    [init]//Create The images that would use to extract the horizontal and vertical lines Mat horizontal = Bw.clone (); Mat vertical = Bw.clone ();//! [init]//!    [Horiz]//Specify size on horizontal axis int horizontalsize = HORIZONTAL.COLS/30; Create structure element for extracting horizontal lines through morphology operations Mat horizontalstructure = Get    Structuringelement (Morph_rect, Size (horizontalsize,1));    Apply morphology operations Erode (horizontal, horizontal, horizontalstructure, point (-1,-1));    Dilate (horizontal, horizontal, horizontalstructure, point (-1,-1)); Show Extracted horizontal lines imshow ("horizontal", horizontal);//! [horiz]//!    [Vert]//Specify size on vertical axis int verticalsize = VERTICAL.ROWS/30; Create structure element for extracting vertical lines through morphology operations Mat verticalstructure = Getstru    Cturingelement (Morph_rect, Size (1,verticalsize)); //Apply morphology operations Erode (vertical, vertical, verticalstructure, point (-1,-1));    Dilate (vertical, vertical, verticalstructure, point (-1,-1)); Show Extracted vertical lines imshow ("vertical", vertical);//! [vert]//!    [Smooth]//Inverse vertical image bitwise_not (vertical, vertical);    Imshow ("Vertical_bit", vertical); Extract edges and smooth image according to the logic//1. Extract edges//2. Dilate (edges)//3. Src.copyto (Smooth)//4. Blur smooth img//5.    Smooth.copyto (src, edges)//Step 1 Mat edges;    Adaptivethreshold (vertical, edges, 255, Cv_adaptive_thresh_mean_c, Thresh_binary, 3,-2);    Imshow ("edges", edges);    Step 2 Mat kernel = Mat::ones (2, 2, CV_8UC1);    Dilate (edges, edges, kernel);    Imshow ("dilate", edges);    Step 3 Mat Smooth;    Vertical.copyto (smooth);    Step 4 blur (smooth, smooth, Size (2, 2));    Step 5 Smooth.copyto (vertical, edges); Show Final result Imshow ("SmooTh ", vertical);//!    [Smooth] Waitkey (0); return 0;}

Explanations and Results1. Load the source image and check that it is loaded successfully, and then display it:

Load the Imagemat src = imread (argv[1]);//Check if image is loaded Fineif (!src.data) cerr << "Problem loading IM Age!!! "<< endl;//Show source imageimshow (" src ", SRC);

2. If the image is not grayscale image converted to grayscale Image:

Transform source image to Gray if it is Notmat gray;if (src.channels () = = 3) {Cvtcolor (src, Gray, Cv_bgr2gray);} else{gray = src;} Show Gray Imageimshow ("Gray", gray);

3. The grayscale image is then converted to two-valued. Note the ~ symbol indicates that we use the inverse version (i.e. bitwise_not):

Apply Adaptivethreshold at the bitwise_not of Gray, notice the ~ symbolmat bw;adaptivethreshold (~gray, BW, 255, CV_ADAP Tive_thresh_mean_c, Thresh_binary, 2);//Show binary imageimshow ("binary", BW);

　　　

4. Before we are ready to apply morphological manipulations to extract the horizontal and vertical lines as a result of separating the notes from the score, but first let us initialize the output image:

　　　

Create the images that would use to extract the horizontal and vertical linesmat horizontal = Bw.clone (); Mat vertical = Bw.clone ();

5. as we have said in theory, in order to extract the objects we want, we need to create the corresponding structural elements. Because here we want to extract the horizontal line, a corresponding structure element has the following shape:　

The code is implemented in the following way:

Specify size on horizontal axisint horizontalsize = horizontal.cols/30;//Create structure element for extracting Hor Izontal lines through morphology operationsmat horizontalstructure = getstructuringelement (Morph_rect, Size ( horizontalsize,1);//Apply morphology operationserode (horizontal, horizontal, horizontalstructure, point (-1,-1)); Dilate (horizontal, horizontal, horizontalstructure, point ( -1,-1));//Show Extracted horizontal Linesimshow (" Horizontal ", horizontal);

6. The use of vertical lines is the same, the corresponding structural elements are as follows:

The code is as follows

Specify size on vertical axisint verticalsize = vertical.rows/30;//Create structure element for extracting vertical Lines through morphology operationsmat verticalstructure = getstructuringelement (Morph_rect, Size (1,verticalsize));// Apply morphology operationserode (vertical, vertical, verticalstructure, point ( -1,-1));d ilate (vertical, vertical, Verticalstructure, point ( -1,-1));//Show Extracted vertical linesimshow ("vertical", vertical);

7. as you can see, you'll notice that the edges of the notes are a bit rough. For this reason, we need to smooth the edges to get smoother results

Inverse vertical imagebitwise_not (vertical, vertical); Imshow ("Vertical_bit", vertical);//Extract edges and smooth Image according to the logic//1. Extract edges//2. Dilate (edges)//3. Src.copyto (Smooth)//4. Blur Smooth img//5. Smooth.copyto (src, edges)//Step 1Mat edges;adaptivethreshold (vertical, edges, 255, Cv_adaptive_thresh_mean_c, Thresh_ BINARY, 3,-2); Imshow ("edges", edges);//Step 2Mat kernel = mat::ones (2, 2, CV_8UC1);d ilate (edges, edges, kernel); Imshow (" Dilate ", edges);//step 3Mat smooth;vertical.copyto (smooth);//step 4blur (smooth, smooth, Size (2, 2));//Step 5smooth.cop Yto (vertical, edges);//Show final resultimshow ("smooth", vertical);

Extracting horizontal and vertical lines based on morphological operations