The Python machine learning Case Series Tutorial--CTR/CVR FM, FFM algorithm __ algorithm

Full Stack Engineer Development Manual (author: Shangpeng)

Python tutorial full solution FM problem Source

CTR/CVR prediction, the user's gender, occupation, education level, category preference, commodity category, etc., after the conversion of one-hot encoding will result in sparse sample data. In particular, commodity category this type of characteristics, such as the end of the product category of about 550, using one-hot code to generate 550 numerical characteristics, but each of these 550 characteristics of a sample, and only one is valid (not 0). Thus, data sparsity is an unavoidable challenge in practical problems.

Another feature of One-hot encoding is that it causes a large feature space. For example, the product category has 550-dimensional features, a categorical feature is converted to 550-D numerical features, and the feature space is dramatically increased.

At the same time, by looking at a large number of sample data can be found that some features after the association, and the relationship between the label will improve. For example, the associated features of "USA" and "Thanksgiving", "Chinese" and "New year" have a positive impact on the user's clicks. In other words, users from "Chinese" are likely to have a lot of browsing and buying behavior in "the New year", while in "Thanksgiving" there will be no special consumer behavior. The positive correlation between this correlation feature and label is common in practical problems, such as "cosmetics" commodity and "female" sex, "ball sports accessories" goods and "male" sex, "movie ticket" Commodity and "movie" category preference, etc. Therefore, it is very meaningful to introduce the combination of two features. FM Fundamentals

A polynomial model is the most intuitive model that contains a combination of features. In the polynomial model, the combination of feature XI X_i and XJ X_j is represented by Xixj X_ix_j, that is, both Xi X_i and XJ X_j are not zero, and the combination feature XIXJ X_ix_j makes sense. From the point of contrast, we only discuss the second-order polynomial model. The expression of the model is as follows

Y (x) =w0+∑i=1nwixi+∑i=1n∑j=i+1nwijxixj (1) y (x) = w_0+ \sum_{i=1}^n w_i x_i + \sum_{i=1}^n \sum_{j=i+1}^n w_{ij} x_i x_j \ta G{1}

where n n represents the characteristic number of the sample, Xi X_i is the value of the first I feature, W0 W_0, WI w_i, Wij w_{ij} are model parameters.

As can be seen from the formula (1), the parameters of the combined feature have N (n