How to Learn about gesture tracking: Realtime and robust hand tracking from depth (4) Hand Initialization

Original paper reading, reproduced please indicate the source: http://blog.csdn.net/ikerpeng/article/details/39251373

After reading this paper many times, I can see why the structure of this article is arranged as follows: put the hand initialization part at the end of all the optimization algorithms in this article, it means that the entire algorithm can be optimized without such a part. We can see from table 2 in the article that the result obtained without hand Initialization is 2mm worse than the result obtained with initialization. If you do not have high requirements for the final result, you can ignore this part.

We found from the beginning of hand initialization that if you use initialization, it should be performed on each frame: re-initialization (in fact, I don't know why the result of the previous frame is not used as the initialization of the next frame ).

1. Review

Let's first review what the initialization of our input data is. First, the data obtained through Intel's creative gesture camera is filtered by the median to remove noise points. Morphological Processing fills in holes. Then, the data is projected into a 3D point cloud. Randomly retrieve 256 points from these point clouds. In this way, the input data is obtained.

The output data is a 48-ball model parameter. We get the parameter solution of the model by solving the cost function.

2. Reflection

Let's think about the problems that exist. First, we randomly select 256 point clouds from them. Such a small amount of data has made the solution less reliable. Second, when there are a lot of shadows and curves in your gestures, the resulting point cloud data is very concentrated, so it is not reasonable to match the 48 model area expanded by God. Therefore, Initialization is necessary (this is my guess ).

3. Initialization Method

The initialization method used in this article is part detection. Specifically, the finger is detected first, and the pose of the hand is initialized with the finger. Therefore, we will explain this in two parts.

Finger Detection:

In this paper, a clever method is adopted: Split a 3D finger into a X-Y finger and a z-finger.

The X-Y finger is the finger parallel to the x-y plane. Then we can obtain a hand mask (that is, remove redundant information from the depth graph through binarization. For example, we set the value of depth to 0 if it is less than M, 1. that is, the points out of this range are removed ). At this time, we Initialize an extreme point on this mask as the starting point, and then calculate the distance from the ground (I think it is the closest distance between the two points on the surface ). When the distance from the ground wire is the farthest, but basically the distance from one finger is almost the same, we think this is a finger and we are extending its size, until you get a finger shape.

Z-finger is considered a point perpendicular to the camera. We can find the local minimum value in the deep image. Then we fill the depth graph with flood fill until we reach a fingertip distance. At this time, we draw two circles (the outer circle is 3 pixels larger than the radius of the sphere and the angle is 60 degrees) to see the ring formed by the two circles (the ring area) the pixels above 90% are regarded as a z-finger.

Hand initialization:

After getting finger, we will use known finger to initialize hand pose (specifically, the equation in the article ).

How to Learn about gesture tracking: Realtime and robust hand tracking from depth (4) Hand Initialization