Important variables for user tiering

According to my previous experience, because there are sensitive data involved, so the use of the virtual and symbolic processing, may not be easy to read, but the digital presentation is not the main theme, this article focuses on the user-layered approach, hoping to help.

This article discusses the user layering, different from the regular user segmentation, the biggest difference is that the hierarchy is the concept of sequencing, that is, the progressive relationship between the layers, and the general subdivision is the concept of the classification, that is, the relatively independent of each other. Broadly speaking, subdivision includes layering.

Now for example market sellers layered, introduced the entire research method. According to the experience of the sellers, we choose the important variables involved in the sellers ' layering, and extract the data from BI 1 million sellers.

Refactoring variables

First, the data of 1 million collection sellers were cleaned and the distribution of the important variables was investigated. Because some fixed distance variables exist beyond the normal range of data, such as the number of transactions, transaction volume, and so on, some of the ordered variables at both ends of the sample too little, such as sellers star, shop type. These are not conducive to the construction of the model, so each to be analyzed variables are subdivided into a number of groups, the principle is: the group as much as possible, and the fixed distance variable more than the distribution of each group as far as possible to ensure that within the normal range, avoid abnormal values. The Adjusted grouping table is slightly.

1 million data were randomly divided into three 600,000 of the database, four data for follow-up analysis, to ensure the stability of the obtained index.

Factor analysis

First of all important variables to do factor analysis, to remove multiple collinearity between variables, after several attempts to eliminate the Alipay transaction volume and transaction results more stable, and more in line with business experience. The final KMO value is 0.788,bartlett spherical test, which is very suitable for factor analysis. Finally, 6 common factors were extracted, GMV and stars, time, order service, the type of Wang Pu, physical goods, whether to participate in the elimination of insurance, the cumulative variance contribution rate is 90.4% (see table below), interpretation effect is very strong; the number of residual >0.05 between the original matrix and the reconstructed matrix is 10. %, the fitting effect is also very good.

After many times, the whole factor analysis was very stable, the cumulative variance contribution rate of the first three factors reached 60%, which was the main factor, and the cumulative variance contribution rate of the three factors was 30%, which was the secondary factor. This is consistent with daily business experience.

Hierarchical

The factor score of each sample can be calculated with six factors, and the proportion of each factor is as follows:

Because the factor synthesis score is a standardized value, using (X-min)/(maximum-minimum), convert it to 0-100 index, and then according to the index level, and the actual situation of the layered, TOP1 as the highest index of 10%,TOP2 for several times the height of 20%, TOP3 refers to a number of high 30%, the remaining is the fourth layer, the lowest index of 40%. The method is simple and the data is stable, which is beneficial to the practical application. The upper and lower limits of each group are as follows, and the upper and lower limits of such division can be fixed and amended gradually.

discriminant analysis

According to the proportion of 7:3 divided into analysis samples and validation samples, using discriminant analysis of the four-layer division results of the test, that is, 6 factors score and hierarchy to do discriminant analysis. The obtained discriminant function has a significant contribution to distinguishing each layer, and the variance contribution rate of the first discriminant function interpretation is 98.2%, the main function. Using the covariance matrix of the group, the correctness rate of the sample and the sample is 91.6%, and the accuracy of the cross verification is high.

There are obvious differences between sellers on important variables, as follows:

To find out the key variables with higher explanatory power from six factors, and to judge the suitability based on the actual business experience, seven key variables were selected. They are directly related to the level of discriminant analysis, the first main discriminant function of the variance contribution rate is 97.8%, using the group of covariance matrix calculation, analysis samples and validation samples of the correct rate, cross-check the correct rate is 85%, also reached a high level.

Regression analysis

In order to facilitate the application and simplify the layered computing process, the seven key variables and factor synthetic score indices identified in discriminant analysis were used to analyze their explanatory power.

Regression analysis showed that R, R Square and adjusted R Square were 0.985, 0.970 and 0.970 respectively, and the remaining standard error was 2.709, which reached a smaller level. D The value of Urbin-watson is 1.252, there is a certain gap with 2, the independence of residual difference is acceptable, comprehensive judgment, the model interpretation effect is very good.

In multiple collinearity, the minimum tolerance value of sellers star is 0.39, and the condition index of eighth principal component is less than 15, which indicates that there is no serious multiple collinearity.

The main indicators of the variables are as follows:

Through the common analysis of the normalized partial regression coefficient and the partial correlation coefficient, we know that the influence of the GMV on the Factor comprehensive score index is more important when the shop is long, the amount of the last three months, and the sellers star.

Factor comprehensive score index =a+b1* sellers star segment +b2* Open shop long segment +b3* Nearly three months GMV Total amount segment + B Vol. Shop type + B 5* whether main kind + B 6* order service number of segments + B 7* whether to participate in the elimination of insurance

Therefore, through these seven key variables Predictor Comprehensive score index is very suitable, after obtaining the new factor comprehensive score index, according to above upper and lower limit adjacent value, may divide the seller level.

In summary, the research process is summarized as follows:

1, according to the previous study of the user's understanding, to determine the importance of participation in the hierarchical variables, the extraction of background data, data cleaning and processing;

2, the factor analysis to participate in the analysis of the important variables to reduce the dimension, calculate the factor synthesis score;

3, the factor synthesis score into the index, according to the distribution of the index, the user layered, and discriminant analysis, the layered results of the verification;

4, according to the explanation of variables and the actual situation of the business, the key variables are selected as the independent variables, the factor synthesis score as the dependent variable, the regression equation is established, and the key variables are used to calculate the comprehensive score, and the rapid layering is convenient for the later business application.

5, the background data randomly split into different databases, respectively, repeat the above analysis process, repeatedly verify the stability of the results.

Reflections on the follow-up study

The whole research is done, perhaps the most valuable is the final regression equation, although the interpretation is very high, but still lack of some difficult to obtain important variables, such as the monthly advertising amount, including through-Train, Diamond booth, the subsequent research will gradually include these variables. This also shows that the user stratification study examines the variables to be as comprehensive as possible, so that the results can be more reference value.

There is the end of the layered results appear "insipid", each layer of sellers in important variables, the basic are strong stronger, weak weaker, characteristics are not obvious. This is also the difference between stratified research and subdivision research, and layering is more the result of trend. Follow-up can try to use the unequal probability of sampling to reduce a number of samples, such as low bills sellers accounted for the vast majority, can be appropriate to reduce this part of the sample, to a certain extent, can balance the role of various important variables in the hierarchy.

No matter which sample structure, it needs to be in the practical application, the test effect, continuously iterative perfect.

Source: http://ued.taobao.com/blog/2010/12/31/user_stratified_study/