Simple and rough fingertip detection method (fingertips detection)

Simple and rough fingertip detection method (fingertips detection)

Zouxy09@qq.com

Http://blog.csdn.net/zouxy09

 

In the field of human-computer interaction, if you can better detect your fingertips, it will greatly improve the richness and flexibility of interaction. At present, there are many fingertip detection methods. I have tried two simple methods here. Both methods use the hand's geometric features, which are simple but rough and not robust enough.

 

Method 1: center of gravity Distance Method

See, red points are the center of gravity of the hand, then the distance between all points on the edge of the hand and the center of gravity is traversed clockwise or counterclockwise, there will be five peaks, five fingers respectively, so that we can simply find them. If you only stick out one or two fingers, you only have one or two peaks.

The simple code is as follows:

1. Perform Gaussian blur on the image;

2. Skin Color Division (the background should not have a skin color. If yes, additional information should be added to eliminate interference );

3. Find the hand contour;

4. Computing moment, that is, center of gravity;

5. Find your fingertips.

 

// Simple fingertips detection // Author: zouxy // Date: 2013-3-23 // homepage: http://blog.csdn.net/zouxy09// Email: zouxy09@qq.com # include "opencv2/opencv. HPP "using namespace CV; using namespace STD; void skinextract (const mat & frame, mat & skinarea); int main (INT argc, char * argv []) {mat frame, skinarea; videocapture capture; capture. open (0); If (! Capture. isopened () {cout <"no camera! \ N "<Endl; Return-1;} while (1) {capture> frame; // mat frame = imread (" fingertips(1).jpg "); If (frame. empty () break; skinarea. create (frame. rows, frame. cols, cv_8uc1); skinextract (frame, skinarea); MAT show_img; frame. copyto (show_img, skinarea); vector <point> contours; vector <vec4i> hierarchy; // find the contour findcontours (skinarea, contours, hierarchy, cv_retr_ccomp, done ); // find the largest contour int inde X; double area, maxarea (0); For (INT I = 0; I <contours. size (); I ++) {area = Hangzhou area (MAT (contours [I]); If (area> maxarea) {maxarea = area; Index = I ;}} // drawcontours (frame, contours, index, scalar (0, 0,255), 2, 8, hierarchy); moments moment = moments (skinarea, true); Point Center (moment. m10/moment. m00, moment. m01/moment. m00); Circle (show_img, center, 8, scalar (0, 0,255), cv_filled); // find the fingertip vector <Po Int> coupoint = s [Index]; vector <point> fingertips; point TMP; int max (0), count (0), notice (0 ); for (INT I = 0; I <coupoint. size (); I ++) {TMP = coupoint [I]; int Dist = (TMP. x-center. x) * (TMP. x-center. x) + (TMP. y-center. y) * (TMP. y-center. y); If (Dist> MAX) {max = DIST; notice = I;} // calculate the number of points maintained by the maximum value. If it is greater than 40 (this value needs to be set, I originally wanted to set it according to the max value, // but it was not successful and I don't know why), so I thought it was the fingertip if (Dist! = Max) {count ++; If (count> 40) {COUNT = 0; max = 0; bool flag = false; // If (center. Y <coupoint [Notice]. y) continue; // For (Int J = 0; j <fingertips. size (); j ++) {If (ABS (coupoint [Notice]. x-fingertips [J]. x) <20) {flag = true; break ;}} if (FLAG) continue; fingertips. push_back (coupoint [Notice]); Circle (show_img, coupoint [Notice], 6, scalar (0,255, 0), cv_filled); line (show_img, center, coup Oint [Notice], scalar (255, 0, 0), 2) ;}} imshow ("show_img", show_img); If (cvwaitkey (20) = 'q') break;} return 0;} // skin color extraction. skinarea is a binarization skin color image void skinextract (const mat & frame, mat & skinarea) {mat YCbCr; vector <mat> planes; // convert to ycrcb Color Space cvtcolor (frame, YCbCr, cv_rgb2ycrcb); // separate multi-channel images into multiple single-channel images (YCbCr, planes); // use the iterator to access the matrix element matiterator _ <uchar> it_cb = planes [1]. begin <uchar> (), it_cb_end = planes [1]. end <uch Ar> (); matiterator _ <uchar> it_cr = planes [2]. begin <uchar> (); matiterator _ <uchar> it_skin = skinarea. begin <uchar> (); // the skin color of a person is in The YCbCr Color Space distribution range: 100 <= CB <= 127,138 <= CR <= 170for (; it_cb! = It_cb_end; ++ it_cr, ++ it_cb, ++ it_skin) {If (138 <= * it_cr & * it_cr <= 170 & 100 <= * it_cb & * it_cb <= 127) * it_skin = 255; else * it_skin = 0;} // expansion and corrosion, expansion can fill the concave hole (bridge the crack), corrosion can eliminate the fine convex ("spot" noise) dilate (skinarea, skinarea, MAT (5, 5, cv_8uc1), point (-1,-1); erode (skinarea, skinarea, MAT (5, 5, cv_8uc1 ), point (-1,-1 ));}

 

Method 2: Curvature Analysis

As you can see, the curvature of the groove between the fingertip and the two fingers is the largest. If the contour of the hand is saved by a series of points {PI}, we can use [Pi, Pi-K]
And [Pi, PI + k]
The inner product of two vectors (DOT multiplication) is used to find the points with high curvature, such as the computation result of Φand β, because the angle between the two vectors tends to be 90 degrees, that is to say, their inner product tends to be 0, so the smaller the inner product result, the larger the curvature, we only need to set a threshold, you can find the tip between the hand and the two fingers.

How can we tell the groove between the fingertip and the two fingers? We can use the cross-multiplication of two vectors for calculation. we imagine that we are in 3D space, and there is another z direction, in beta, the cross multiplication of the two vectors is greater than 0 (the cross multiplication direction is vertical, the paper faces you, And the zaxis is positive ), while the cross multiplication at the diameter is smaller than 0 (The vertical side of the Cross plane is far away from you, and the negative side of the Z axis ).

In summary, by constructing two vectors at each point, we can determine the fingertip by comparing their dot multiplication and cross multiplication.

The simple code is as follows:

1. Perform Gaussian blur on the image;

2. Skin Color Division (the background should not have a skin color. If yes, additional information should be added to eliminate interference );

3. Find the hand contour;

4. Find your fingertips.

 

// Simple fingertips detection using curvature determination // Author: zouxy // Date: 2013-3-23 // homepage: http://blog.csdn.net/zouxy09// Email: zouxy09@qq.com # include "opencv2/opencv. HPP "using namespace CV; using namespace STD; void skinextract (const mat & frame, mat & skinarea); int main (INT argc, char * argv []) {mat frame, skinarea; videocapture capture; capture. open (0); If (! Capture. isopened () {cout <"no camera! \ N "<Endl; Return-1;} while (1) {capture> frame; // mat frame = imread (" fingertips(1).jpg "); If (frame. empty () break; gaussianblur (frame, frame, size (3, 3), 0); skinarea. create (frame. rows, frame. cols, cv_8uc1); skinextract (frame, skinarea); MAT show_img; frame. copyto (show_img, skinarea); vector <point> contours; vector <vec4i> hierarchy; // find the contour findcontours (skinarea, contours, hierarchy, cv_retr_ccomp, Cv_chain_approx_simple); // find the maximum contour int index; double area, maxarea (0); For (INT I = 0; I <contours. size (); I ++) {area = Hangzhou area (MAT (contours [I]); If (area> maxarea) {maxarea = area; Index = I ;}} // drawcontours (frame, contours, index, scalar (0, 0,255), 2, 8, hierarchy); moments moment = moments (skinarea, true); Point Center (moment. m10/moment. m00, moment. m01/moment. m00); Circle (show_img, center, 8, scal AR (0, 0,255), cv_filled); // find the fingertip vector <point> coupoint = s [Index]; int max (0), count (0 ), notice (0); vector <point> fingertips; point P, Q, R; For (INT I = 5; (I <(coupoint. size ()-5) & coupoint. size (); I ++) {q = coupoint [I-5]; P = coupoint [I]; r = coupoint [I + 5]; int dot = (Q. x-P. x) * (Q. y-P. y) + (R. x-P. x) * (R. y-P. y); If (DOT <20 & dot>-20) {int cross = (Q. x-P. x) * (R. y-P. Y )-(R. x-P. x) * (Q. y-P. y); If (Cross> 0) {fingertips. push_back (p); Circle (show_img, P, 5, scalar (255, 0, 0), cv_filled); line (show_img, center, P, scalar (255, 0, 0), 2) ;}} imshow ("show_img", show_img); If (cvwaitkey (20) = 'q') break;} return 0 ;} // skin color extraction. skinarea is a binarization skin color image void skinextract (const mat & frame, mat & skinarea) {mat YCbCr; vector <mat> planes; // convert to ycrcb Color Space cvtcolor (frame, YCbCr, cv_rgb2ycrcb ); // Separate multi-channel images into multiple single-channel images (YCbCr, PLANeS); // use the iterator to access the matrix element matiterator _ <uchar> it_cb = planes [1]. begin <uchar> (), it_cb_end = planes [1]. end <uchar> (); matiterator _ <uchar> it_cr = planes [2]. begin <uchar> (); matiterator _ <uchar> it_skin = skinarea. begin <uchar> (); // the skin color of a person is in The YCbCr Color Space distribution range: 100 <= CB <= 127,138 <= CR <= 170for (; it_cb! = It_cb_end; ++ it_cr, ++ it_cb, ++ it_skin) {If (138 <= * it_cr & * it_cr <= 170 & 100 <= * it_cb & * it_cb <= 127) * it_skin = 255; else * it_skin = 0;} // expansion and corrosion, expansion can fill the concave hole (bridge the crack), corrosion can eliminate the fine convex ("spot" noise) dilate (skinarea, skinarea, MAT (5, 5, cv_8uc1), point (-1,-1); erode (skinarea, skinarea, MAT (5, 5, cv_8uc1 ), point (-1,-1 ));}

 

Effect:

 

For more information, see the following resources:

Finger-detection-and-gesture-recognition [Code]

Hand and finger Detection Using javacv [project]

Hand and fingers detection [Code]