Deep Research Institute Digital Image Processing second big job: Fruit automatic recognition (2) HSV spatial clustering and SIFT algorithm target recognition

The same as the usual, directly posted report ~

Programe list:

Programe was developed in the condition of Windows aswell as Linux server, programming language is Matlab (www.mathworks.c OM).

classify.m, kmeans.m: function for K-means clustering.

main_kmeans.m: main function for K-means clustering.

hsvfeatureextraction.m: Extracting the HSV features for a picture.

hsvfeaturesorting.m: Sorting the extracted HSV features.

hsvmed.m,hsvmed_accuracy.m: Making the medclassifcation in HSV space and calculate its classfication accuracy.

sift.m,match.m, showkeys.m: function for Siftalgorithm.

MainG1 G3 g5.m: Making recognition among specific groupsand calculating its recognition accuracy (by The Times ex Amine).

Methodology:

The images is from ten kinds of fruits and each fruit contains 10images. First we dicided to use SIFT algorithm for all of the fruitsrecognition but the images is too big to process and even on SUP Er computerserver provided by Sun Yat-sen University. Hence, we decided to make a advanceclassification by colors then apply SIFT among the same groups.

Full algorithm flow diagram is as follows:

Fig1. Algorithm Flowdiagram

Firstly, perior to the color feature extraction, region of interest (ROI) extraction are the basic step for image processing , here we applied an edgedetection method called watershed. Algorithm of watershed is a segement methodbased on topology theory. The basic concept is regarding the image as alandform of a map, altitude is defined by the gray level at that pixel. Local Minimalvalue and its related region called Catchment Basin and its boundry calledwatershed. More specific, sketch map is shown in fig.2:

Fig.2 Sketch Map ofwatershed

To calculate thewatershed in a image, iterative algorithm should be applied; Core Algorithmsare sorting and submerging. Firstly gray levels of each pixel is sort in theforward direction. Then ' wate ' r is ' poured ' on the image and watershed willappear. ROI extraction was taken place in Hue of HSV space and the Extractionresult are shown in fig.3:

Fig.3 ROI Extractionresult of watershed method

As we can see in fig.3,watershed method can derive feasible results.

RGB color space is firstly applied for the color classfication feature,however, result by this feature is bad and then W E decided to does thecalssfication in HSV space.

By extracting the HSV features, we can draw these features of eachpicture in HSV color space and calculated the mean Featu Re of each group:

Fig.4 HSV features of imagesand groups in HSV space

As is shown in Fig.4, redpoints represent the HSV feature of each image and blue points represent THEHSV feature of each G Roup. We can conclude this pictures in same group is incorrelation with each other and has difference from pictures form Diffe Rentgroups. Then MED is applied for each images and try to assign them intospecific group.

fig.5 classifcation withmed in HSV color space

However, finalclassification result is not feasible, only 67% pictures is assigned in ToRight catalog. Then we decided to expand the size of groups and try to make theclassification perfect as well as less members in groups F Or a betterperformance of post-sift process.

K-means Clustering Methodwas applied to those pictures, firstly the number of seed points is both and Wecan get the Correc T result but each group had a pictures which cancause lots of trouble in further SIFT process. So the number of seed pointsstarted to grow until clustering error happens, finally we can get five groupscontain differen T kinds of fruits:

Fig.6 Results of fruitsclassfication in HSV color space

After Classification,group2 and Group 4 contain only one kind of fruit which means classificationcan get 100% accuracy for The recognition of these, fruits, and Group1 andGroup3 contain three kinds of fruits and Group 5 contain 2 kinds of fr Uits.

In the folder '/testsample ', g1-g4 only left one picture for giving the example and G5 is the ROIs ofdifferent both kinds o F fruits which is ' guava ' and ' green apple ', in order toindentify these the kinds of fruits of similar color, SIFT was applied.

SIFT is the Algorithmwhich aims at detecting local features, by detecting the interest points of oneimage and descriptors of scale and orientation then making the match amongdifferent pictures, a feasible result can be obtained.

fig.7 Flow diagram Ofsift Matching

by Examing SIFT in thesame groups to obtain the accuracy, test samples were generated by random andother pictures is used for training Sampls. After time of the random checkout,the highest identification probability can be 93%, which are acceptable for ourdaily use.

Furter Works can aim atthe better efficiency of ROI extraction and grouping, plastic bags is terriblefor texture feature Extraction, testing on texture generated a bad result. Ifwe can remove the influence of plastic bags, I think texture features would giveus some interesting results.

Reference:

Digital Image Processing (MATLAB edition), second edition of Gonzales

Lowe SIFT Original:

Http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf

The pattern recognition method of applying SIFT operator in MATLAB:http://blog.csdn.net/abcjennifer/article/details/7372880

Sift algorithm explained:

http://blog.csdn.net/abcjennifer/article/details/7639681

K-means algorithm explained:

Http://www.cnblogs.com/moondark/archive/2012/03/08/2385770.html

Watershed Segmentation algorithm:

http://blog.csdn.net/zd0303/article/details/6703068

Post code:

%NAME:HSV feature, Extraction%function:extract ROI feature from HSV images%author:changle Zhang, [email protect Ed]clc;clear All;close all;%i           Nitilizationpathname= ' E:\Pro2_15S158746_ Zhang Zhangle \fruit Samples for project2\ Snow White Apple '; % Modify folder name CD (pathname);d irs=dir ([, ' *.jpg ']);d Ircell=struct2cell (dirs) '; Filenames=dircell (:, 1); Hsvmean=zeros (10,3             ); % pre-allocated average parameter for Number=1:10rgbimg=imread (char (filenames (number))), R=rgbimg (:,:, 1); G=rgbimg (:,:, 2); B=rgbimg (:,:, 3); Grayimg=rgb2gray (rgbimg); HSVIMG=RGB2HSV (rgbimg); H=hsvimg (:,:, 1); S=hsvimg (:,:, 2); V=hsvimg (:,:, 3);%rgb = Imread (' pears.png ');% Read original image i = s;% converted to grayscale image hy = fspecial (' Sobel ');%sobel operator hx = hy '; Iy = IMFilter ( Double (i), HY, ' replicate '),% filter for y-direction edge Ix = IMFilter (double (i), HX, ' replicate '),% filter for X-direction Edge gradmag = sqrt (ix.^2 + iy.^2); To touch%3. Mark the foreground and background separately: this example uses morphological reconstruction techniques to mark the foreground object, first using the open operation, after opening the operation can be removed some small targets. SE = Strel (' disk ', 20);% circular structure element io = imopen (i, se);% morphology open operation Ie = Imerode (i, se);% corrosion of image iobr = Imreconstruct (Ie, i);% Morphological reconstruction IOC = Imclose (Io, se);% morphologyOff Operation IOBRD = Imdilate (iobr, SE);% expands the image iobrcbr = Imreconstruct (Imcomplement (IOBRD), Imcomplement (IOBR));% Morphological reconstruction iobrcbr = Imcomplement (IOBRCBR),% image negation bw = IM2BW (IOBRCBR, Graythresh (IOBRCBR)),% conversion to two value image%figure;%imshow (BW),% display binary image%title (' thresholded opening-closing by Reconstruction ') bww=double (BW); HSVIMG (:,:, 1) =bww.*hsvimg (:,:, 1); Hsvimg (:,:, 2) =bww. *hsvimg (:,:, 2); Hsvimg (:,:, 3) =bww.*hsvimg (:,:, 3);%figure;%imshow (rgbimg); imgsize=size (grayimg); counter=0;hsv=[0 0 0];hsv=double (HSV); for I=1:imgsize (1) for j=1:imgsize (2) if BWW (i,j) >0 HSV (1) =HSV (1) +double (HSVI            MG (i,j,1));            HSV (2) =HSV (2) +double (hsvimg (i,j,2));            HSV (3) =HSV (3) +double (hsvimg (i,j,3));        counter=counter+1;             End Endendhsvmean (number,:) =hsv/counter;endallhsvmean=mean (Hsvmean); Save (' Hsvmean.txt ', ' Hsvmean ', '-ascii ');       % Save (' allhsvmean.txt ', ' Allhsvmean ', '-ascii ') of the HSV mean parameter information in each image's ROI; % save entire class ROI HSV mean parameter information

%NAME:HSV feature Med%function:get the Accury for a HSV MED results%author:changle Zhang, [email protected]clc; Clear All;close all;%i Nitilizationload chsvfeature.mat;pathname= ' E:\cl\DIP\Pro2_15S158746_ Zhang Zhangle \pro2_15s158746_ Zhang Zhangle \fruit Samples for           Project2 '; % ^^ Change this to make analysis among NC or psd/or ADCD (pathname);d Irs=dir;dircell=struct2cell (dirs) '; Filename=dircell (    3:12,1); Results=cell (11,10); for Num=1:10 CD (char (num));    Results (11,num) =filename (num);    Hsvs=load (' Hsvmean.txt ');        For I=1:10 Test=hsvs (i,:);        Distance=zeros (10,1);        For j=1:10 distance (j) = (Test (1)-features (j,1)) ^2+ (Test (2)-features (j,2)) ^2+ (Test (3)-features (j,3)) ^2;        End Position=find (distance==min (distance));    Results (I,num) =filenames (position); End CD (pathname); Endcounter=0;for i=1:10 for J=1:10 if sum (Size (char (results)) 1,i (char (==size (results))) ) ==2 if sum (Size (char (results (1,i))) ==size (char (resuLTS (j,i))) ~=0 counter=counter+1;                 End End endendaccury=counter/100; %save (' hsvresults.txt ', ' results ');

%name:maing5%function:get the Accury for SIFT in group 5%author:changle Zhang, [Email protected]clc;clear all;c           Lose all;% Initialize pathname= ' E:\cl\DIP\Pro2_15S158746_ Zhang Zhangle \pro2_15s158746_ Zhang Zhangle \testsample\g5 '; % ^^ Change this to make ANALYSISCD (pathname);d irs=dir ([, ' *.jpg ']);d Ircell=struct2cell (dirs) '; Filenames=dircell (:, 1)            ; accury=0;for i=1:100 Testsample=round (20*rand);            Test=char (Filenames (testsample));    alla=0;        allb=0;            For num=1:10 if num~=testsample [Numa (num), M1,m2]=match (Test,char (Filenames (num)));        alla=alla+m2;        End End A=sum (NUMA)/alla;            For num=11:20 if num~=testsample [Numb (num), M1,m2]=match (Test,char (Filenames (num)));        allb=allb+m2;    End End B=sum (numb)/allb;        If a>=b test= ' guava ';    Testnum=1;        End If A<b test= ' green apple ';    testnum=2; End if ((testnum==1) && (testsample<=10)) | | ((testnum==1) && (testsample>=10)) accury=accury+1; Endendaccury

% num = Match (image1, image2) percent this function reads-images, finds their SIFT features, and% displays lines Connectin  G The matched keypoints. A match is accepted% only if It distance are less than distratio times the distance to the% second closest match.% It Returns the number of matches displayed.%% example:match (' scene.pgm ', ' book.pgm '); function [num,m1,m2] = match (Image1, IM Age2)% Find SIFT keypoints for each image[im1, des1, Loc1,m1] = SIFT (image1); [Im2, Des2, loc2,m2] = sift (image2);% for efficiency in Matlab, it's cheaper to compute DOT products between% Unit vecto  Rs rather than Euclidean distances. Note that the ratio of% angles (ACOs of dot products of the unit vectors) are a close approximation% to the ratio of euclide An distances for small angles.%% distratio:only keep matches in which the ratio of vectors angles from the% nearest to S   Econd nearest neighbor is less than distratio.distratio = 0.62; % for each descriptor in the first image, select it match to SEcond image.des2t = Des2 ';        % Precompute Matrix Transposefor i = 1:size (des1,1) dotprods = des1 (i,:) * DES2T;  % computes vector of dot products [vals,indx] = sort (ACOs (dotprods));   % take inverse cosine and sort results% Check if nearest neighbor have angle less than distratio times 2nd.   if (Vals (1) < Distratio * Vals (2)) match (i) = indx (1);   else Match (i) = 0; endend% Create A new image showing the images side by side.% im3 = Appendimages (im1,im2);% Show a figure with lines Jo Ining the accepted matches.%figure (' Position ', [Size (im3,2) size (im3,1)]);% ColorMap (' Gray ');% Imagesc (IM3);% Hold on;% cols1 = size (im1,2), percent for i = 1:size (des1,1)% if (match (i) > 0)% Line ([Loc1 (i,2) loc2 (Match (i), 2) +cols 1], ...% [Loc1 (i,1) loc2 (Match (i), 1)], ' Color ', ' C '),% end% end% hold off;num = SUM (Match > 0); fprintf (' Foun D%d matches.\n ', num);d isplay (m2)

% [Image, descriptors, locs] = Sift (imagefile) percent of this function reads a image and returns its Sift keypoints.% Input par ameters:% imagefile:the file name for the image.%% returned:% image:the image array in double format% desc  Riptors:a k-by-128 Matrix, where each row gives a invariant% descriptor for one of the K keypoints. The descriptor is a vector% of all values normalized to unit length.% locs:k-by-4 Matrix, with which each row h  As the 4 values for a% KeyPoint location (row, column, scale, orientation). The% orientation is in the range [-pi, PI] radians.%% credits:thanks for initial version of this program to D. A Lvaro and% J.J. Guerrero, Universidad de Zaragoza (modified by D. Lowe) function [image, descriptors, locs,m] = s IFT (imagefile)% Load imageimage = Rgb2gray (Imread (imagefile)),% Imshow (image,[]);%%%%%%%%%%%%%%%%%%%%%% If you have the Image processing Toolbox, you can uncomment the following% lines to AlloW input of color images, which'll be converted to grayscale.% if ISRGB (image)% image = Rgb2gray (image);% End[rows, CO LS] = size (image); % Convert into PGM imagefile, readable by "keypoints" Executablef = fopen (' tmp.pgm ', ' w '); if f = =-1 Error (' Could not ' C reate file TMP.PGM. '); endfprintf (F, ' p5\n%d\n%d\n255\n ', cols, rows); Fwrite (f, image ', ' uint8 '); fclose (f);% call Keypoints Executableif Isunix command = '!.  /sift '; else command = '!siftwin32 '; endcommand = [command ' <TMP.PGM >tmp.key '];eval (command);% Open Tmp.key and Check its Headerg = fopen (' Tmp.key ', ' r ') and if g = =-1 Error (' Could not open file Tmp.key. '); End[header, Count] = fscanf (g, '%d%d ', [1 2]); if Count ~= 2 error (' Invalid KeyPoint file beginning. '); Endnum = Header (1), Len = header (2), If Len ~= error (' KeyPoint Descriptor length Invalid (should be 128). end% creates the output matrices (use known size for efficiency) Locs = double (Zeros (num, 4));d escriptors = double (zero S (num, +));% Parse Tmp.keyfor i = 1:num [vector, Count] = fscanf (g, '%f%f%f%f ', [1 4]);    %row Col scale Ori if Count ~= 4 error (' Invalid keypoint file format ');        End Locs (i,:) = vector (1,:);    [Descrip, Count] = fscanf (g, '%d ', [1 Len]);    if (count ~=) error (' Invalid keypoint file value. ');    End% Normalize each input vector to unit length descrip = descrip/sqrt (sum (descrip.^2)); Descriptors (i,:) = Descrip (1,:); endm=size (descriptors,1); fclose (g);

% Showkeys (image, locs) percent This function displays a image with SIFT keypoints overlayed.% Input parameters:% Image: The file name for the image (grayscale), Locs:matrix in which per row gives a KeyPoint location (row,% col Umn, scale, orientation) function Showkeys (image, locs) disp (' Drawing SIFT keypoints ... ');% Draw image with Keypointsfigur E (' Position ', [image,2] Size (image,1)]), ColorMap (' Gray '), Imagesc (image), hold on;imsize = size (image);    1:size (locs,1)% Draw an arrow, each line transformed according to KeyPoint parameters.    Transformline (Imsize, locs (i,:), 0.0, 0.0, 1.0, 0.0);    Transformline (Imsize, locs (i,:), 0.85, 0.1, 1.0, 0.0); Transformline (Imsize, locs (i,:), 0.85, -0.1, 1.0, 0.0); Endhold off;%------subroutine:transformline-------% Draw the GI Ven line in the image, but first translate, rotate, and% scale according to the KeyPoint parameters.%% parameters:% Arra ys:% imsize = [rows columns] of image% KeyPoint = [SUBPIxel_row subpixel_column scale orientation]%% scalars:% x1, y1; Begining of vector% x2, y2; Ending of Vectorfunction transformline (imsize, KeyPoint, x1, y1, x2, y2)% the scaling of the unit length arrow is set to a Pproximately the radius% of the region used to compute the keypoint Descriptor.len = 6 * KeyPoint (3);% Rotate the Keypoi NTS by ' ori ' = KeyPoint (4) s = Sin (KeyPoint (4)), C = cos (KeyPoint (4)),% Apply TRANSFORMR1 = KeyPoint (1)-Len * (c * y1 + s) * x1); c1 = KeyPoint (2) + Len * (-S * y1 + c * x1); r2 = KeyPoint (1)-Len * (c * y2 + S * x2); c2 = KeyPoint (2) + Len * (- S * y2 + c * x2); line ([C1 C2], [R1 R2], ' Color ', ' C ');

% drawing with load (' Hsvfeature.mat '); A=features;plot3 (A (:, 1), A (:, 2), A (:, 3), ' B. '); hold Onpathname= ' E:\cl\DIP\Pro2_ 15s158746_ Zhang Zhangle \pro2_15s158746_ Zhang Zhangle \fruit Samples for Project2 ';           % ^^ Change this to make analysis among NC or psd/or ADCD (pathname);d Irs=dir;dircell=struct2cell (dirs) '; Filenames=dircell (3:12,1); Results=cell (11,10); for Num=1:10    CD (char (filenames (num)));    Hsvs=load (' Hsvmean.txt ');    PLOT3 (Hsvs (:, 1), Hsvs (:, 2), Hsvs (:, 3), ' R. ');    Hold on    CD (pathname); end

Deep Research Institute Digital Image Processing second big job: Fruit automatic recognition (2) HSV spatial clustering and SIFT algorithm target recognition