Feature Engineering (supplement)--feature combination

Feature combination change is also a means of feature selection, this part of the work can play a space to see your imagination and experience. The combination of changes here is far beyond the subtraction of existing features (such as kernel tricks).

To give a more imaginative example-the recommended algorithms for "people you probably know" in social networks now on the market are almost always based on complementary networks, so the recommended person may simply complement and refine the circle of friends, and the recommended person may be uninteresting, which can lead to a poor recommendation and a loss of interest in the referral. Stanford Little handsome Professor Jure Leskovec in a 2010 article "predicting Positive and negative Links in Online social Networks" spoke of a recommendation based on user feedback "you might know. People "recommendation algorithm, he put a total of 16 positive and negative feedback of the triangle relationship between the adjacent three people as a feature vector to express the positive and negative feedback between user A and the recommended target user C, to remove some of the known positive and negative feedback side to build training data, with simple logistic The regression training model achieves a good result.

Then we could not help asking a question, whether to choose the full feature set, the model is the highest accuracy rate, if not, then exactly what kind of feature set to choose the most accurate rate.

Here is a diagram, the horizontal axis is the number of selected features, longitudinal axes is the accuracy of cross-validation obtained, from which can be seen, not all features selected, the highest accuracy, when a few characteristics can be the highest accuracy when the choice of features more, but the superfluous.