OpenCV using python--to apply statistical skin tone model and skin color segmentation relative to the Origin Energy of the block

Apply a statistical skin tone model and skin color segmentation relative to the block Origin Energy 1. Skin Color Segmentation Introduction (1) A brief introduction to the statistical skin color model

In the previous article, we used the training data to successfully calculate the posterior probability P (cs|v) of the training data, that is, the probability that the known pixel value determines the color of the skin. We also obtained a skin mask with a posterior probability greater than the different threshold values. However, the skin-tone detection of the skin-color model has not been given before, because the test effect depends on the RGB color space distribution of the pixel in which the training data corresponds. This article will give the application effect of the statistical skin color model, and improve the defects of the single use model.

(2) Introduction to block Origin Energy

The skin blocks that are split in the image are collectively referred to as blocks. According to the statistical skin color model in a picture may be obtained thousands of blocks, then the corresponding will have thousands of blocks of the center, the center of the block is called the Block origin. The energy in relation to the origin of the block is composed of two parts: the static energy, which is defined as two distances from the pixel to the origin of the block, and the other part is the dynamic energy, which is defined as the difference between the posterior probability of the pixel value and the posterior probability of the pixel with equal energy. The equivalent energy is the experimental set value, in the upper and lower bounds of the posterior probability of skin tone, if the posterior color of a pixel is greater than the equivalent energy value, the energy saturation of the pixel point, set to 1. At the same time, if the posterior color of the pixel is less than the lower bound of the posterior skin color, the energy of the pixel point is null and set to 0. If the posterior skin rate of the pixel is within the range of the lower and equal energy values of the posterior skin color, the calculated pixel energy depends on the weighted sum of the static energy and the dynamic energy.

So what is the significance of the existence of the energy of block and block origin for statistical skin segmentation? Statistical skin color Segmentation depends on the training data. If the training data is too perfect, then the offline learning will be very slow, while in reality most of the time with the light effect is often less likely to collect complete training data, and the more complete data will cause the background color of the more severe interference. So we need to collect the full skin tone training data for limited skin color detection, to achieve the perfect complexion detection in limited occasions. Here is just such a mention, the author from the online database casually find a database to do, temporarily did not consider so many problems, if later used will also back to modify training data. In most complex environments, skin tones are more or less affected by the background color of the environment. So here's the need to manipulate blocks: if you have skin blocks and background blocks at the same time in your skin-color segmentation, be sure to try to get rid of the background blocks. Training database is incomplete will be split out of the skin block is more trivial, and the background block will cause skin color block more trivial, the final extraction of skin block effect must be small pieces and fragmented, such skin segmentation in late gestures or face tracking can not be used at all, so there is a block of the original point of energy demand, Compared with the blind morphological calculation results are more complete and accurate, because the single morphological calculation does not consider the relationship between skin tone and background noise.

So what is the relationship between skin tone and background noise? The color blocks of the statistical model are more complete than the background noise block, and in most cases the skin area is larger (if there is a large skin color background, any segmentation algorithm that does not depend on the foreground position of the skin color is invalidated). So here's a strong hypothesis: assume there's no more skin color background than a skin tone foreground. Then the application of skin color segmentation based on the energy of the Block origin must be: a small area of skin color or a large area of skin color or a complex background without skin color foreground segmentation .

2. Code implementation (1) Read the CSV file

The posterior probability of the skin color class is read from the color mask on the upper bound of the posterior skin rate in the CSV file and the three-channel color values in the 18-bit RGB color space.

############################################################################### #print ' Read skin posterior Probability from csv file ' csvfile = open (' Skin_posterior_thresh.csv ', "RB") reader = Csv.reader (csvfile) # Skin Mask Accord ing to training skins Databasemaskskin = Np.zeros (CRange, np.uint8) # skin posterior probability according to training skins Databaseprobskin = Np.zeros (CRange, np.double) # prepare an empty image Spaceimgskin = Np.zeros (Img.shape, np.uint8) ImgSki Nenergy = Np.zeros (Img.shape, np.uint8) # Copy Original Imageimgcontour = Imggray.copy () Imgthreshblob = imggray.copy () img Energy = imggray.copy () # dataType = 0:skinpix (skin mask); DataType = 1:p (cs| V) DataType = 0rowNum = 0for row in Reader:print len (row) Colnum = 0 for col in Row:if Datat                                    Ype = = 0: # Restore Skin Mask maskskin[(rowNum-1) * CRange + colnum] = Col elif DataType = = 1: # Restore Posterior prObability probskin[(rowNum-1) * CRange + colnum] = decimal.                                             Decimal (col) Colnum + = 1 RowNum + = 1 DataType + = 1

(2) Get mask image

Read the position of the original image in the RGB color space; Obtain the color image with the upper bounds of the corresponding skin color (the skin color is the original, the non-skin part is black), and obtain the initial contour gray image of the skin mask with the upper bound of the skin-color posterior probability;

(3) Detect skin tone block

Finds the contours of the initial contour grayscale image and obtains the area's maximum ascent line, while covering the skin color block with all the detected skin tones ( red block, segmented result including background skin noise) on the skin tone mask color image.

############################################################################### #print ' blob detection ' # Find the Contours in the Maskcontour,_ = Cv2.findcontours (Imgcontour.copy (), Cv2. Retr_external,cv2. chain_approx_simple) # Print num of contoursprint '%d contours found. '% len (contour) # find Maximum Areamaxarea = 0.0for cn TIDX in range (len (contour)):        # each contour        cnt = contour[cntidx]    # contour Area Area    = Cv2.contourarea ( CNT)        If area > Maxarea:        maxcontour = contour[cntidx]        maxarea = Areaprint ' maximum area: ', Maxareaprint ' co Ntour length before: ', Len (contour) # Loop over the contoursfor C in contour:    # Draw all red contours    cv2.drawconto Urs (imgskin,[c],-1, (251,37,26), Cv2.cv.CV_FILLED)

(4) Remove background skin color block

Remove the background color block that is less than 1% of the maximum skin color block area (the background color block refers to skin tones or skin tones that contain a large number of background areas, and may contain a small amount of trivial skin blocks). After the small skin patches have been removed, the smaller skin tones are less, but the area of the skin that originally belonged to the same skin color unit is more trivial. The deleted background skin color block remains a red block, and the non-deleted skin color block is covered with a blue block.

# blob Origins Eliminationprint ' eliminate false positive area ' CNTIDX = 0while cntidx < len (contour):            cnt = Contour [Cntidx]    # Contour Area Area    = Cv2.contourarea (CNT)        if area < 0.01 * Maxarea:        del contour[cntidx]        # cntidx Decre  ASE 1 after elimination    else:        # Print Area        cntidx + = 1print ' contour length after: ', Len (contour)                      # Loop Over the contoursfor C in contour:    # Draw relative Big Blue contours    cv2.drawcontours (imgskin,[c],-1, 41,207,1 (+), Cv2.cv.CV_FILLED) # Draw biggest green contour cv2.drawcontours (imgskin,[maxcontour],-1, (120,236,64), cv2.cv.CV_ FILLED)

(5) Block Origin expansion

Before the block origin is expanded, you need to save the center (that is, the block origin) of the block corresponding to the deleted background skin color block, and calculate the distance between the pixel points and the block origin. Then, we can calculate the energy of each pixel that satisfies the lower bound of the posterior probability of skin tone according to the rules of energy function, and then generate the energy image after mapping to the gray interval. The pixel points that satisfy the lower limit of the posterior probability of skin tone meet the energy threshold and expand to the pixel on the skin color block.

############################################################################### #print ' blob spread ' for    R in Range (rows): For    C in range (cols): Energy                = 0        # V (P) > Pth_blob (BLOB acceptance threshold)        if imgthres Hblob.item (r,c) = = 255:                             # Get values from RGB color space            r = Img.item (r,c,0)/4            G = Img.item (r,c,1)/4            b = Img.item (r,c,2)/4                        color = B * * + + G * + R                                    VP = Probskin[color]                        # distance between V (p) and BL OB origin            E0 = 1.0/(Distance[r * cols + c] + + 0.0000001) Energy                        = E0 + K * (VP-PEQ) Energy                         = min (1, MA X (energy,0))                    Imgenergy.itemset ((r,c), energy * 255)

(6) Detect skin tone blocks and remove background color blocks to detect skin tones in the energy image, and delete the corresponding background color block that is less than the maximum contour area area 3%. The purpose is to remove potentially expanded background color blocks. GreenBlock is the largest skin tone block after the expanded contour area.
3. Experimental results

The experiment uses all the images from the previous articles to try to avoid the perfect illusion caused by the ideal image, while exposing the imperfect image where the above steps improve.

The first image has a small skin color foreground area, a large skin color background area and serious noise disturbance. So there are possible problems with the results of the experiment: skin color background area larger than skin color foreground area will result in background and foreground swaps under strong assumptions. In fact, the result is not as serious as it might be.

Start from the beginning to look at the effect of the image analysis: the contour image obtained by the statistical skin tone model is the most trivial result of the skin color foreground in these columns, but the skin tone background noise is smaller than the parametric skin tone model. The maximum area contour is found in the skin color image as Alice's right arm, the temporary result is acceptable because at least not the contour area corresponding to the background skin color. The threshold block image is a skin mask image corresponding to the lower bound of the posterior probability of the statistical skin-color model, because it is the nether of probability, and it contains the skin tones between the upper bound of the probability, so that the complexion foreground and background block are more plump than the contour image. The energy image is the energy in the image mapping (the more white means the higher the energy, the gray area is less easy to see, indicating that the probability is in the upper and lower boundary between the skin block less, that is, the large number of uncertain skin color block). Compared to the skin image before denoising, the skin color foreground and background area are more connected and complete, the maximum area contour area is Alice's left and right arm, the segmentation result is more complete.

Answer the previous question: Why is the color foreground and background swap problem in the image not as serious as it might be? The background and foreground distance in the image is far away, and the saturation of the background color is diluted, so the distance from the average skin color is farther than the real one, and with the distant object receiving multi-angle illumination, the color distribution on the surface of the object is more complex, and the skin background becomes trivial when split. The final calculation of contour area will be small.

The second image has more skin color prospects, but Alice's skin is very small and tight. Therefore, the expansion of the block origin because the pixel distance from the block origin is too close, resulting in the calculation of the static energy weight is too large, so that the final non-skin area is also expanded. A satisfactory effect can be achieved by reducing the weight factor of the static energy properly, but if the weight factor is excessively reduced, the full face will not be expanded. Considering the existence of the strong hypothesis and the desire of my laziness, there is no experiment on the value of weighting coefficients.

The color foreground and background of the third original image are very complete. But the statistical skin color model is not the same training data, so the statistical skin model segmentation results are not good, but finally quite complete. In addition to the complexion of the human face is not as changeable as the chameleon, that is, the area in the color space is limited in scope, so there are some robustness to different training data.

When you visualize the skin, the black blocks of the non-skinned areas in the face of the latter two pictures (the largest area of the skin color foreground) are blocked by green blocks. Because the fill color uses the FloodFill () function that comes with it, and does not make a mate between the fill function. OK ~ The feeder content is put on first.

Conclusion

In addition to the above methods, it can also be done with a parametric skin tone model. Visually, the area is divided in the color space, and then the mapping from color space to the possibility of skin tone is given. But this mapping is not easy to find directly. But the statistical skin model can be used without naive Bayes, and other statistical methods can be tested.

OpenCV using python--to apply statistical skin tone model and skin color segmentation relative to the Origin Energy of the block