machine learning techniques and algorithms

MySpace qizmt is a mapreduce framework designed to run and develop distributed computing application projects running on Windows Server large-scale clusters. MySpace qizmt is an open-source framework initiated by MySpace to develop trustworthy, scalable, and super-Simple distributed application projects. Open Source Address: http://code.google.com/p/qizmt /. Infer. NET is an open-source framework that runs Bayesian inference in graphical mode. It is also used for ProbabilityProgramDesign. Open

(First chapter above)1.2.5 Linalg Linear Algebra LibraryBased on the basic operation of matrices, the Linalg Library of NumPy can satisfy most linear algebra operations.. determinant of matrices. Inverse of the Matrix. Symmetry of matrices. The rank of the matrix. The reversible matrix solves the linear equation1. Determinant of matrices from Import * in[#N-order matrix determinant operation in [6]: A = Mat ([[[1,2,3],[4,5,6],[7,8,9]]) in [print]det (A):"6.66133814775e-162. Inverse of the Matrix

Decision tree is to select the most information gain properties, classification.The core part is to use information gain to judge the classification performance of attributes. The information gain is calculated as follows:Information entropy:Multiple categories are allowed.Calculates the information gain for all attributes, choosing the largest root node as the decision tree. Then, the sample branches, continuing to determine the remaining properties of the information gain.Information gain has

resultIf it is an engineering program, consider here if the error rate=0 case, do a special deal.In the end, Lin theoretically discussed the basis of AdaBoost:Why does this approach work?1) The Ein may be getting smaller with each step of the way2) enough sample size, VC bound can ensure that Ein and eout close (good generalization)Lin then introduces a classic example of a adaboost:To find a weak classifier, that is no weaker than the one-dimension stump, but it is so weak classifier, through

ReferenceNB: High efficiency, easy to implement;LR: Less assumptions about data, strong adaptability, can be used for online learning, and the requirement of linearDecision tree: Easy to interpret, independent of data linearity or not; easy overfitting, no online supportRF: Fast and scalable, with few parameters, possibly over fittingSVM: High accuracy, processing of non-linear sub-data (high-dimensional data processing); Memory consumption, difficult

This section is about the nuclear svm,andrew Ng's handout, which is also well-spoken.The first is kernel trick, which uses nuclear techniques to simplify the calculation of low-dimensional features by mapping high-dimensional features. The handout also speaks of the determination of the kernel function, that is, what function K can use kernel trick.In addition, the kernel function can measure the similarity of two features, the greater the value, the

algorithm to initially estimate the number of K.2) How to choose the initial K pointsThe common algorithm is random selection. But often the effect is not very good, also can be similar to the method, the line uses the hierarchical clustering algorithm to divide the K clusters, and uses these clusters ' centroid as the initial centroid.3) method of calculating distancesCommonly used such as European distance, cosine angle similarity degree.4) Algorithm Stop conditionThe maximum number of iterat

other.Suppose we choose the attribute R as the split attribute, DataSet D, R has K different values {v1,v2,..., Vk}, so d according to the value of R into K-group {d1,d2,..., Dk}, after splitting by R, the amount of information required to separate the different classes of DataSet D is:information gain is defined as before and after the split, two of the amount is only poor:The following example uses Python to illustrate a decision tree construct using the information gain method:The main steps

application thread exists in the contents of the set logs, and modify the corresponding remembered sets, this step needs to pause the application, parallel running.Survival Object calculation and cleanup ( Live Data counting and Cleanup )It should be noted that in G1, it is not that final marking pause is executed, it is certain to perform cleanup this step, because this step needs to suspend the application, G1 in order to achieve quasi-real-time requirements, It is necessary to reasonably pla

Logistic regression is used to classify, and linear regression is used to return.Linear regression is the addition of the properties of the sample to the front plus the coefficients. The cost function is the sum of squared errors. Therefore, in the minimization of the cost function, you can directly derivative, so that the derivative equals 0, as follows:Gradient descent can also be used to learn the same gradient as the logistic regression form.Advantages of linear regression: simple calculatio

Continuous update ...1.k-Nearest Neighbor algorithmAdvantages: High precision, insensitive to outliers, no data input settingsCons: High computational complexity, high spatial complexityApplicable data range: Numerical and nominal typeApplicable scenarios:2.ID3 Decision Tree AlgorithmAdvantages: The computational complexity is not high, the output is easy to understand, the missing middle value is not sensitive, can process the irrelevant characteristic dataDisadvantage: May cause over-matching

Extremely light of a semester finally passed, summer vacation intends to learn the big step down this machine learning techniques.The first lesson is the introduction of SVM, although I have learned it before, but I heard a feeling is very rewarding. The blogger sums up a ballpark figure, and the specifics areTo listen: http://www.cnblogs.com/bourneli/p/4198839.htmlThe blogger sums it up in detail: http://w

This is what we have learned (except decision tree)Here is a typical decision tree algorithm, with four places to choose from:Then introduced a cart algorithm: By decision Stump divided into two categories, the criterion for measuring subtree is that the data are divided into two categories, the purity of these two types of data (purifying).The following is a measure of purity:Finally, when to stop:Decision tree may be overfitting, reducing the number of Ein and leaves (indicating the complexity

Course Address: https://class.coursera.org/ntumltwo-0021. What are the motivations of neural networks (nnet)?A single perceptron (Perceptron) model is simple, limited in capability and only linearly segmented. It is easy to implement logic and, or, non, and convex sets by combining the perceptual machine model, but it is not possible to achieve the XOR operation and the ability is limited. Multi-level perceptual m