Introduction to the ASM (Active Shape Models) Algorithm

ASM is a feature extraction method based on the statistical learning model. This method was proposed in 95 years, but it is still a good method for extracting facial feature points. Tim Cootes, the author of the method, later proposed the AAM algorithm, which is also very famous. However, if it is simple to locate the feature points, I think the effect of ASM may be better. Like most statistical learning methods, ASM also includes train and test (or fit), that is, shape modeling build and shape matching fit. This algorithm is actually very simple and can be used for real-time detection. However, I have been reading it for a long time, and there are still some problems to understand. There are certainly many mistakes. I hope you can correct them more.

1. Build a model 1. select an appropriate feature

It is important to select feature points. Good feature points should probably meet the following requirements: edge points, curvature large, T-type connection points, and other points on the line of the above points. (For example)

Record the order of these feature points to facilitate subsequent determination of the border and line of the image. This is also the advantage of the ASM method, and the obtained feature points are ordered. In this way, a set of feature set X is obtained.

We will regard it as a 2n vector.

2. Shape statistical model

The obtained 2n-dimension vector has a high dimension (because there are usually many feature points in the image), but these 2n dimensions are highly correlated with each other, the distance between each other is roughly unchanged. Here, the author uses the most commonly used PCA for dimensionality reduction and extracts the principal component. Here, the author calls it modes. Therefore, any set of feature points can be seen as a coordinate point in the space of the principal component vector, and this coordinate point is considered to be the mean of these points, so that any point is the coordinate origin plus a vector. That is:

Here, P contains the covariance matrix of the first t principal component, and B is a T-dimensional Vector used to control the shape changes of feature points. When B is 0, X is the coordinate origin (average shape), which is actually a good understanding. But here B should not be too big. If it is too large, it will cause the whole shape to change beyond the scope of the face change, which will make people abnormal, everyone understands... So the author makes constraints here

There is another problem in this step, that is, the number of principal components t In PCA is selected, there are two ways: 1) Select the first s; 2) Select the first K % of the component, the second method is recommended.

3. Match the model with the new point set

This process is the core of the ASM method. To put it simply, it is nothing more than rotating, shrinking, and translating the just-obtained model.

T indicates the rotation, contraction, and translation matrix.

Our goal is to make model X and image feature set y the closest, where the objective function selection is the Euclidean distance and minimum of each point.

The specific matching process is to first initialize the B vector to 0 to obtain model X and use other methods to find the transformation matrix T. This is very simple. You can use other methods (Haar face detector, optical flow method, kalman filtering, etc.) roughly find the target location, size, and direction. Then, use y to reverse B and update B until convergence. It is easy to fall into the local minimum, which is also a disadvantage of this method.

2. Matching (FIT) of models in Images)

To put it bluntly, we can find the matching point of the model in the image. As we have already said, we can use other methods to roughly find the location and size direction of the model (do not underestimate this step. If preprocessing is not good, is easy to fall into the local minimum), the next step is how to improve the parameter to get a good matching result. Parameters that can be adjusted in the model include B, XT, yt, S, and theta. The author mentioned two methods:

1) The prior information assumes that the feature points are mainly the Strong Edge Points of the image, and the gray gradient of the image follows the Gaussian distribution, as long as the maximum gradient value near the model feature point is found, that is, the location where the feature points are located.

As follows:

This method is obvious and sensitive to image noise, which is not recommended here.

2) sample the texture information around the model feature points, compare the texture of the image and the model training set, and find the closest texture point, that is, the feature point. The texture comparison tool here is the Markov distance. In order to improve the search efficiency, the author also mentions the pixel textures around the multi-resolution search. For rough scales, the search range is large, and detailed searches for detailed scales, this improves the matching efficiency. I recommend this method.

As for the code, I have found many versions, including opencv-based and Matlab-based versions. You can find the corresponding link on the ASM page of the Wiki, but I will do it myself and summarize it, I am not very familiar with MATLAB, and the amount of code is too large. I gave up learning. Asmlib-opencv is written by a Chinese student abroad. It has source code, but the effect is average. There is also asmlibrary, a very good work of Chinese people, but the core part only provides Lib, his method works well, the Code style is also very good, real-time is also very good, thank you for your guidance. Another cool man wrote stasm which is not based on opencv. The Code volume is too large and I gave up. As for some other works, there is only demo and no code, which is a pity. It would be appreciated if any of you could provide the source code of opencv with good ASM results.