kaggle machine learning datasets

instrumental permutation test (permutation test) in the use of statistics in RF is used to measure the importance of feature items. n samples, D dimensions per sample, in order to measure the importance of one of the features di, according to permutation test the N sample of the di features are shuffled shuffle, shuffle before and after the error subtraction is the importance of this feature. RF often does not use permutation Test during training, but instead disrupts the OOB feature it

Discovery modeThe linear model and the neural network principle and the goal are basically consistent, the difference manifests in the derivation link. If you are familiar with the linear model, the neural network will be well understood, the model is actually a function from input to output, we want to use these models to find patterns in the data, to discover the existence of the function dependencies, of course, if the data itself exists such a function dependency. There are many types of

++ = 1.0 currline = line. strip (). split ('\ t') linearr = [] For I in range (21): linearr. append (float (currline [I]) If int (classifyvector (Array (linearr), trainweights ))! = Int (currline [21]): errorcount + = 1 errorrate = (float (errorcount)/numtestvec) print 'the error rate of this test is: % F' % errorrate return errorratedef multitest (): numtests = 10; errorsum = 0.0 for K in range (numtests): errorsum + = colictest () print 'after % d iterations the average error rate is: % F' %

90avg/total 0.82 0.78 0.79 329The accuracy of gradient tree boosting is 0.790273556231 Precision recall f1-score support 0 0.92 0.78 0.84 239 1 0.58 0.82 0.68 90avg/total 0.83 0.79 0.80 329Conclusion:Predictive performance: The gradient rise decision tree is larger than the random forest classifier larger than the single decision tree. The industry often uses the stochastic forest c

First, cross-validation.Cross-validation (validation) is an evaluation of statistical analysis, machine learning algorithms for data sets independent of the training data generalization ability (generalize), can avoid overfitting problems.Cross-validation generally needs to be as satisfying as possible:1) The proportion of the training set should be enough, generally more than half2) uniform sampling of tra

{\partial \mathcal{l}}{\partial B} =0 \rightarrow \sum_{i=1}^{n}\alpha_{i}y_{i}=0\)Bringing these two results back to $ \mathcal{l} (W,b,\alpha) $ gets the following result:\ (-\frac{1}{2}\sum_{i=1}^{n}\sum_{j=1}^{n}\alpha_{i}\alpha_{j}y_{i}y_{j}x_{i}^{t}x_{j}+\sum_{i=1}^{n}\ alpha_{i}\)(2) After getting the above formula, we find that the Lagrangian function contains only one variable, that is \ (\alpha_{i}\), then we can go to the optimal problem:\[\max\limits_{\alpha} \space\-\frac{1}{2}\sum_

noise in the activities as a regularizer). Presumably, for an implicit unit that uses a logical function, its output must be between 0 and 1, and now we use a binary function in the forward direction instead of the logic function in the hidden unit, the random output 0 or 1, the output is computed. Then in the reverse, we use the correct method to do the correction. The resulting model may have a poor performance on the training set, and the training speed is slower, but its performance on the

rowsCluster_assment = Mat (Zeros ((M, 2)) # A column of record cluster index values, a column of storage errorsCentroids = Rand_cent (Data_set, k) # Generate random centroidcluster_changed = TrueWhile cluster_changed:cluster_changed = FalseFor I in Xrange (m): # Calculates the distance from each data point to the centroidMin_dist = infMin_index =-1 # Calculate the minimum distance from each data point to the centroid for J in Xrange (k): Dist_ji = Dist_eclud ( Centroids[j,:], data_set[i,:]

Data Set Classification in machine learning with supervised (supervise), datasets are often divided into two or three groups: the training set (train set) validation set (validation set) test set. The training set is used to estimate the model, the validation set is used to determine the network structure or the parameters that control the complexity of the mod

minimizing the degree of impurity at each step, the cart can handle the outliers and be able to handle the vacancy values. The termination condition of the tree partition: 1, the node achieves the complete purity; 2, the depth of the tree reaches the depth of the user3, the number of samples in the node belongs to the user specified number;Pruning method of tree is a pruning method of cost complexity;See details: http://blog.csdn.net/tianguokaka/article/details/9018933 Copyright NOTICE: This ar

: matplotlib Annotation Matplotlib provides an annotation tool annotations, which can be used to add text annotations to data graphs. Annotations are usually used to interpret data. I didn't understand this code, so I only gave the code in the book. #-*-Coding: cp936-*-import matplotlib. pyplot as pltdecisionnode = dict (boxstyle = 'sawtooth ', Fc = '0. 8 ') leafnode = dict (boxstyle = 'round4', Fc = '0. 8 ') arrow_args = dict (arrowstyle =' The index method is used to find the index returne

The decision tree extracts a series of rules from the data collection, which can be represented by a flowchart, whose data form is very easy to understand, and the decision tree is often used in expert systems.1, Decision tree Construction: ① uses the ID3 algorithm (the highest information gain) to divide the data set; ② recursively creates a decision tree.2, using the matplotlib annotation function, you can transform the stored tree structure into easy-to-understand graphics.3. The Pickle modul

AboutDTC =Decisiontreeclassifier () $ #Training - Dtc.fit (X_train, Y_train) - #Predicting saved results -Y_predict =dtc.predict (x_test) A + " " the 4 Model Evaluation - " " $ Print("accuracy:", Dtc.score (X_test, y_test)) the Print("Other indicators: \ n", Classification_report (Y_predict, Y_test, target_names=['died','survived'])) the " " the accuracy: 0.7811550151975684 the Other indicators: - Precision recall F1-score support in the died 0.91 0.78 0.84 236 the survived 0.58 0.80 0.67 Abo

-frequent, then all its superset (the collection containing the collection) is also infrequent. The advent of the Apriori principle, after knowing that some itemsets are non-frequent, does not need to calculate the superset of the set, effectively avoids exponential growth of the number of itemsets, and calculates frequent itemsets within a reasonable time.2. RealizeApriori algorithm is a method of discovering frequent itemsets. of the Apriori algorithmTwo input parameters are minimum support le