best data mining

Data preprocessing STEP1: Data sampling: Because in the establishment of customer churn model process, the loss of customers often accounted for the proportion of all customers are very small, at this time, the best way is to retain the entire loss of customer population, but not the loss of customer population sampling, so that customer churn and non-customer churn in the 1:1~1:2 STEP2:

Distinction between classification and clustering classification (classification): A classifier will "learn" from the training it receives, thus having the ability to classify unknown data, a process typically called supervised learning (supervised learning). The so-called classification, in simple terms, is based on the characteristics of the text or attributes, divided into the existing categories.Common classification algorithms include: Decision t

I. Concepts Association Rule Mining: discovering interesting and frequent patterns, associations, and correlations between item sets of a large amount of data, such as the food database and relational database. Measurement of the degree of interest of association rules:Support,Confidence K-item set: a set of K items Frequency of the item set: number of transactions that contain the item set Frequent Item Se

transaction by user shell+ip+ hostname according to different user's login (all three are the same user) Based on this, the basic principle of mining 2 algorithm for user input command sequence frequent pattern is realized. The fp-growth algorithm mainly solves the collection of frequent items where the number of occurrences reaches a certain threshold in multiple sets. A FP tree is a compressed representation of input

(' relative importance ') Plt.draw () plt.show () The code is a bit long, but mainly divided into two, one is model training, the other is based on the importance of training to screen important features and drawing. The attributes that are more important than 18 are obtained as shown in the following illustration: It is important to see the three properties of TILTLE_MR title_id gender. and the title related to the attributes are our analysis of the name, can be seen in some string propertie

frequent itemset (frequent itemsets) Called I={i1,i2,..., im} i=\{i_1, i_2, ..., i_m\} is a collection of items (item) , D={t1,t2,..., Tn} d=\{t_1, T_2, ..., t_n\}, I∈[1,n] I∈[1,n For the transaction dataset (Transaction data itemsets), the transaction Ti t_i consists of several items in I i. Set S S as a set of items, s={i|i∈i} s=\{i|i∈i\}, short term set (itemset). The set of items that contains K items is called the K-item set . T T is a transacti

Several basic concepts and two basic algorithms for association rules are described in the previous few. But actually in the commercial application, the writing algorithm is less than, understands the data, grasps the data, uses the tool to be important, the preceding basic article is to the algorithm understanding, this article will introduce the open source utilizes the

only 1. So the count of conditional pattern bases is determined by the minimum count of nodes in the path.Depending on the conditional pattern base, we can get the conditional FP tree for that commodity, for example i5:According to the conditions of the FP tree, we can do a full array of combinations, to get the frequent patterns excavated (here to the commodity itself, such as i5 also counted in, each commodity mining out of the frequent pattern mus