Stanford machine learning online course notes

ArticleDirectory

• 10 debugging a Learning Algorithm:
• 11 trading off precision and recall:
• 13. outlier detection

Http://siliangdu.tk/2011/11/106)
10 debugging a Learning Algorithm:

1. Get more training sets: corrected over-fitting

2. Use fewer features: corrected over-fitting

3. Use more features: Correct bias (relative to over-fitting, which should be due to insufficient fitting)

4 using polynomial features (x1 ^ 2, X1 * x2. ..): Corrected bias

5. Reduce the penalty factor lambda: Modify bias

6. Increase penalty factor: Correction of over-fitting

11 trading off precision and recall:

Precision: the number of samples predicted to be positive (that is, number 1) is actually positive, true positive/(true positive + false positive ).

Recall: In a positive sample, the number of correct predictions is positive. True positive/(true positive + false negtive)

Increase the threshold (that is, h (x)> = 0.7 is predicted as positive, h (x) <0.7 is predicted as 0, 0.7 is the threshold), then increase the precision and reduce the recall; on the contrary, reduces the threshold value of judgment, reduces recall, and increases precision.

12 SVM

Increase C: Modify bias

Reduced C: corrected over-fitting

When using Gaussian Kernel:

Increase σ: The image slope slows down and overfitting is corrected.

σ reduction: The image slope is steep, and bias is corrected.

13. outlier detection

In my opinion, the outlier detection is similar to some specific categories (two types), but there are many differences, mainly reflected in:

1) the training set, cross validation set, and test set of outlier detection are extremely unbalanced, and even only single-class. The training set of classification should be balanced.

2) during cross-validation, classification accuracy is not applicable to outlier detection because of unbalanced datasets.

3) There can be many types of outliers. That is to say, you cannot determine the specific types of outliers. You can only determine that a certain vertex is not a mainstream vertex, and the classification can determine the specific category of each vertex.

Common density-based detection methods, using the Gaussian distribution formula, when the probability p (x) of a vertex x <ε, This vertex is marked as an outlier.

Assume that point X has n features: X (1) x (2 )... x (n), P (x) = p (x (1) * p (x (2 ))*... P (x (n ))

Step: 1) Select the representative feature X (I) for the outlier. 2) Calculate the corresponding feature mean and variance. 3) Calculate p (x)