Playing card Recognition Based on aforge. NET Framework)

Copyright: November 2012

Original article address: Click to View Details

Author: Nazmi Altun

 

Download source code-148.61 KB

Download demo-
3.1 MB

 

Introduction

(Words on the image: Square 4, square J, and black peach 2)

With a robot coupled with a poker card recognition system, you can play the role of a dealer or a human player in 21.1 class poker games. Implementing such a program is also a good way to learn computer vision and pattern recognition.

The aforge. NET Framework technologies involved in this article include binarization, edge detection, affine transformation, blob processing, and template matching algorithms.

It should be noted that the system described in this article is designed for Anglo-American poker and may not be applicable to other types of poker. However, this article describes the basic methods of poker detection and recognition. Therefore, the specific recognition algorithm needs to be changed according to the characteristics of playing cards.

Here is a video demonstration.

(It has been moved to Youku-byoy)

Http://v.youku.com/v_show/id_XNDAxNzQyOTYw.html

Copyright: November 2012

Poker Detection

We need to detect the poker objects on the images (the captured video images, the same below-byoy note) so that we can proceed with the next recognition. To complete the detection, we use some Image filters to process the video.

Step 1: remove the image color (that is, grayscale-byoy injection ). Color removal is an operation to convert a color image into an 8-bit image. We need to convert a color image to a gray image for binarization.

After converting a color image into a gray image, we perform binarization. Binarization (threshold) is the process of converting grayscale images into black and white images. This article uses the Otsu Method for global threshold.

Bitmap temp = source. clone () as bitmap; // copy the original image filterssequence seq = new filterssequence (); seq. add (grayscale. commonalgorithms. bt709); // Add the grayscale filter seq. add (New otsuthreshold (); // Add the binarization filter temp = seq. apply (source); // apply a filter.

(Words on the image: original image, gray image, binary (black and white) image)

With a binary image, you can use the Blob processing method to detect playing cards. We use the blobcounter class of aforge. Net to complete this task. This class uses the connected area Labeling Algorithm to collect statistics and extract independent objects in the image (that is, playing cards-byoy injection ).

// Extract blob blobcounter with a width and height greater than 150 from the image. Extractor = new blobcounter (); extractor. filterblobs = true; extractor. minwidth = extractor. minheight = 150; extractor. maxwidth = extractor. maxheight = 350; extractor. processimage (temp );

After the above code is executed,BlobCounterClass filters out (removed) The blobs (blob, which is the image block blob and independent objects in the image) whose width and height are not between [150,350] pixels. The following part will be renamed as a map block-byoy note ). This helps us to differentiate other objects in an image (if any ). Depending on the test environment, we need to change the filter parameters. For example, if the distance between the ground and the camera increases, the playing cards in the image become smaller. In this case, we need to change the minimum, maximum width, and height parameters accordingly.

Now, we can callextractor.GetObjectsInformation()Obtain the information of all the graph blocks (edge points, rectangular area, center point, area, integrity, and so on ). However, we only need the edge point of the graph block to calculate the center point of the rectangular area and call the pointscloud. findquadriteralcorners function to calculate it.

Foreach (BLOB in extractor. getobjectsinformation () {// obtain the edge point list of playing cards <intpoint> edgepoints = extractor. getblobsedgepoints (BLOB); // Use edge points to locate the four corners of the original image list <intpoint> corners = pointscers. findquadrilateralcorners (edgepoints );}

(Words on the image: Draw Edge Points on the image and look for the corners of each poker player)

After finding the four corners of the playing card, we can extract the normal playing card image from the original image. It can be seen that playing cards can be placed horizontally. It is very easy to detect the horizontal placement of playing cards. After playing cards are put down, we know that the card height is greater than the width, so if the width of the extracted (converted) image is greater than the height, the card must be placed horizontally. Then, we use the rotateflip function to rotate the playing card to its normal position.

Note that for correct identification, all poker players should have the same size. However, given the different camera angles, the poker card size will change, which may easily lead to recognition failure. To prevent this problem, we adjusted all transformed poker images to 200x300 (pixels.

// Used to extract the playing card quadrilateraltransformation quadtransformer = new week (); // used to adjust the playing card size resizebilinear resizer = new resizebilinear (cardwidth, cardheight); foreach (BLOB in Extractor. getobjectsinformation () {// obtain the poker edge point list <intpoint> edgepoints = extractor. getblobsedgepoints (BLOB); // Use edge points to locate the four corners of the original image list <intpoint> corners = pointscers. findquadrilateralcorners (edgepoints); bitmap cardimg = quadtransformer. apply (source); // extract the playing card image if (cardimg. width> cardimg. height) // If the cardimg is placed sideways. rotateflip (rotatefliptype. rotate90flipnone); // rotating cardimg = resizer. apply (cardimg); // normalize (reset size) Playing cards .....}

(Words on the image: cards extracted from the original image using the quadriteraltransformation class. This class uses the four corners of each card for conversion .)

So far, we have found the four corners of each playing card on the original image, extracted the playing cards from the image, and adjusted them to a uniform size. Now, we can start recognition.

Copyright: November 2012

Recognize playing cards

There are several techniques used to identify playing cards. This article uses the card type (such as the shape on playing cards) and template matching technology. The color and size of playing cards are identified separately. We can enumerate:

public enum Rank {     NOT_RECOGNIZED = 0,     Ace = 1,     Two,     Three,     Four,     Five,     Six,     Seven,     Eight,     Nine,     Ten,     Jack,     Queen,     King } public enum Suit {     NOT_RECOGNIZED = 0,     Hearts,     Diamonds,     Spades,     Clubs }

We will also create the following card class to represent the cards recognized. This category includes the card size, color, extracted poker image, and its four corners on the original image.

Public class card {// variable private rank; // size private suit; // color private bitmap image; // extracted image private point [] corners; // four-point // attribute public point [] Corners {get {return this. corners ;}} public rank {set {This. rank = value ;}} public suit {set {This. suit = value ;}} public bitmap image {get {return this. image ;}// constructor public card (Bitmap cardimg, intpoint [] cornerintpoints) {This. image = cardimg; // set aforge. the intpoint array is converted to system. drawing. point array int Total = cornerintpoints. length; corners = new point [total]; for (INT I = 0; I <total; I ++) {This. corners [I]. X = cornerintpoints [I]. x; this. corners [I]. y = cornerintpoints [I]. Y ;}}}

Copyright: November 2012

Color Recognition

There are four types of standard playing cards: Black peach, plum blossom, square and red peach. The square and red peach are red, and the black peach and plum blossom are black. In addition, the width of the square is greater than that of the red peach, and the width of the plum blossom is greater than that of the black peach. These two features can help us identify colors.

Recognize colors

First, we start with recognizing colors. Correct identification of colors will help us eliminate the other two colors. We will identify the color by analyzing the upper right corner of the playing card image. (The author stressed that this article is based on the specific poker model he selected, and is related to printing and card design-Nobi note)

public Bitmap GetTopRightPart() {     if (image == null)         return null;     Crop crop = new Crop(new Rectangle(image.Width - 37, 10, 30, 60));      return crop.Apply(image); }

(Words on the image: crop the upper right corner of the poker image, and crop the bottom of the previous image again)

After cropping the upper right corner of the playing card, we get a 30x60 pixel image. However, the image contains both the color and size. Because we only analyze the color, we crop the lower half and get a 30x30 pixel image.

Now we can traverse the total number of red and black pixels in the image. If the red component of a pixel is greater than the sum of the blue component and the green component, the pixel is considered red. If the red, green, and blue components are less than 50, and the red components are not greater than the blue and green components, the pixel is considered black.

Char color = 'B'; // start, lock pixel bitmapdata imagedata = BMP. lockbits (New rectangle (0, 0, BMP. width, BMP. height), imagelockmode. readonly, BMP. pixelformat); int totalred = 0; int totalblack = 0; unsafe {// statistics of red and black try {unmanagedimage IMG = new unmanagedimage (imagedata); int Height = IMG. height; int width = IMG. width; int pixelsize = (IMG. pixelformat = pixelformat. format24bpprgb )? 3: 4; byte * P = (byte *) IMG. imagedata. topointer (); // For (INT y = 0; y 

 

Note:Bitmap.GetPixel()The function runs slowly, so we use pointers to traverse pixels.
Distinguish between character cards and number cards
After recognizing the color, we need to determine whether the playing card is a character card. The card numbers are J, Q, and K. There is a prominent feature between the character card and the number card, that is, the number card has a lot of Color Symbols to indicate its size, and the character card is very recognizable, its face has a character avatar. We can simply set a large color shape to analyze poker, rather than using a complex template matching algorithm for it. In this way, recognition of digital cards can become faster.
It is very easy to find out whether a playing card is a character card or a digital card. There are big figures on the character card, but no number card. If we perform edge detection and graph block (BLOB) processing on the card and find the largest graph block, we can determine whether it is a character card or a number card from the size of the block.
Private bool isfacecard (bitmap BMP) {filterssequence commonseq = new filterssequence (); commonseq. add (grayscale. commonalgorithms. bt709); commonseq. add (New bradleylocalthresholding (); commonseq. add (New differenceedgedetector (); bitmap temp = This. commonseq. apply (BMP); extractbiggestblob extractor = new extractbiggestblob (); temp = extractor. apply (temp); // extract the largest image if (temp. width> BMP. width/2) // If the width is greater than the general width of the entire card, return true; // character card return false; // number card}

Therefore, we constantly perform grayscale conversion, local threshold, and edge detection on poker images. Note that we use local thresholds instead of global thresholds to eliminate the problem of poor lighting (that is, when the light is switched, the automatic white balance of the camera will cause the screen to be flickering-Nobi injection ).

(The words on the image are the same as those on the upper and lower cards): original poker image, grayscale, Bradley local threshold, edge detection, and largest image extraction)
As you can see, the biggest chart of a character card is almost as big as the whole playing card, which is easy to distinguish.
As mentioned above, for performance considerations, we will use different recognition technologies to identify character cards and digital cards. For a digital card, we extract the largest image block and recognize its width and color.
private Suit ScanSuit(Bitmap suitBmp, char color) {      Bitmap temp = commonSeq.Apply(suitBmp);      //Extract biggest blob on card      ExtractBiggestBlob extractor = new ExtractBiggestBlob();      temp = extractor.Apply(temp);  //Biggest blob is suit blob so extract it      Suit suit = Suit.NOT_RECOGNIZED;       //Determine type of suit according to its color and width      if (color == 'R')         suit = temp.Width >= 55 ? Suit.Diamonds : Suit.Hearts;      if (color == 'B')         suit = temp.Width <= 48 ? Suit.Spades : Suit.Clubs;       return suit; }


The maximum error of the above test is 2 pixels. In general, because we have adjusted the poker card size to X pixels, the test results will be the same size.
There is no image of the largest suit similar to a digital card on the character card, only the small flower color on the corner. This is why we crop the upper right corner of the poker image and apply the template matching algorithm to identify the color.
The project resource file contains a binarization template image. (See the source code of the project-byoy note)
Aforge. Net also provides a class called exhaustivetemplatematching, which implements an exhaustive template matching algorithm. This class completely scans the original graph and compares each pixel with the corresponding template. Although the performance of this algorithm is poor, we only use it in a small area (30x60) and do not have to worry too much about the performance.
Private suit scanfacesuit (bitmap BMP, char color) {bitmap clubs, diamonds, spades, hearts; // color TEMPLATE 4 // load template resource clubs = playingcardrecognition. properties. resources. clubs; diamonds = playingcardrecognition. properties. resources. diamonds; spades = playingcardrecognition. properties. resources. spades; hearts = playingcardrecognition. properties. resources. hearts; // use the 0.8 similarity threshold to initialize the template matching class exhaustivetemplatematching templatematching = new exhaustivetemplatematching (0.8f); suit = suit. not_recognized; If (color = 'R') // if it is red {If (templatematching. processimage (BMP, hearts ). length> 0) suit = suit. hearts; // match the IF (templatematching. processimage (BMP, diamonds ). length> 0) suit = suit. diamonds; // match the square} else // if it is black {If (templatematching. processimage (BMP, spades ). length> 0) suit = suit. spades; // match the IF (templatematching. processimage (BMP, clubs ). length> 0) suit = suit. clubs; // match plum blossom} return suit ;}

 

(The word on the image: above is, template match? Yes. Below is the template match? No)
Of course, the template cannot match the sample by 100%, so we use the similarity threshold of 0.8 (80%.
Recognize size
The identification size is similar to the identification color. It also identifies character cards and digital cards separately. Because a digital card can be identified only by calculating the number of color blocks on the card surface without template matching, a simple image filter can be used to complete the task.
The scanrank function shown below filters small graph blocks (smaller than 30 pixels long or wide) and calculates the remaining number of graph blocks.
Private rank scanrank (Bitmap cardimage) {rank Rank = rank. not_recognized; int Total = 0; bitmap temp = commonseq. apply (cardimage); // apply the filter blobcounter = new blobcounter (); blobcounter. filterblobs = true; // filter the small block blobcounter. minheight = blobcounter. minwidth = 30; blobcounter. processimage (temp); Total = blobcounter. getobjectsinformation (). length; // obtain the total number Rank = (rank) total; // convert to a large or small (Enumeration type) return rank ;}


(The words on the image: edge detection, filtering the picture blocks with width and height less than 30 pixels, the total number of remaining pictures is 10, that is, the number of cards)
Therefore, digital cards can be identified without template matching algorithms or OCR. However, for character cards, we need to use template matching again for identification.
Private rank scanfacerank (bitmap BMP) {bitmap J, K, Q; // character TEMPLATE 4 // load resource J = playingcardrecognition. properties. resources. j; k = playingcardrecognition. properties. resources. k; q = playingcardrecognition. properties. resources. q; // initialize exhaustivetemplatematching templatematchin = new exhaustivetemplatematching (0.75f) with 0.75; Rank = rank. not_recognized; If (templatematchin. processimage (BMP, J ). length> 0) // J Rank = rank. jack; If (templatematchin. processimage (BMP, K ). length> 0) // K Rank = rank. king; If (templatematchin. processimage (BMP, q ). length> 0) // Q Rank = rank. queen; return rank ;}


Because of the difficulty in identification, we use 0.75 (75%) as the similarity threshold this time.
Known issues
The implementation of this article can only identify separate playing cards (no overlap-byoy note ). Another known problem is that changes in the light environment often cause recognition errors.
Copyright: November 2012
Conclusion
The image use cases used in this article are from the aforge. NET Framework. Aforge. NET provides many useful features for developers in the Machine Vision and machine learning fields. It is also very simple for me.
This article can also be improved, for example, how to identify a card before it is placed separately. Another improvement is the use of this system for AI players.
 
History
* 7th, Oct., 2011: First Draft
License
This article and the source file code and files that come with it follow the code plan website open source license (cpol)
 
About the author
Nazmi Altun
Softeare developer
Turkey
 
Copyright: November 2012
 
(Full text)