The way to Big data processing (Gibbs sampling)

A: Introduction and ways to learn

(1) Gibbs sampling (Gibbs sampling) and related algorithms ( learn good places for Gibbs sampling, EM, MCMC algorithms, etc.)

1) Recommend you read Bishop pattern recognition and machine learning, speak very clear, partial theory some;

2) Read artificial intelligence,2, 3 version, all have;

3) If the English is good, the most convenient is to check Wikipedia, this is the most clear (graduate recommended to read this)

4) Do not Baidu to go, Baidu in the academic aspect of the current do very poor; Google academic very good, Google technical issues are also very good, foreign open source sites and communities more, not as the country's motionless let you register, let you pay.

(2) Introduction:

The popular point of explanation, sampling is a certain probability distribution, see what happened. For example--a can only E: Eat, study, play, Time T: Morning, afternoon, evening, the weather W: Sunny, windy, rain. Now for a sample, this sample can be: Play + afternoon + Sunny

The problem is that we don't know P (e,t,w), or that we don't know the joint distribution of three things。 Of course, if you know, there is no need to use Gibbs sampling. but we know three things about the conditional distribution。 In other words, p (e| T,W), p (t| E,W), p (w| E,T). Now all we have to do is to get the joint distribution by the three known distribution of conditions and the Gibbs sampling method.

Specific methods. First, casually initialize a combination, i.e. study + Night + wind, then change one of the variables according to the conditional probability. Specifically, suppose we know that at night + windy, we give e a variable, for example, learn-"eat." We can change the next variable according to the conditional probability, and turn the evening into morning based on learning + wind. Similarly, turning the wind into wind (and of course the same variable). So learn + night + windy-"eat + Morning + wind."

The same method, get a sequence, each unit contains three variables, that is, a Markov chain. Then skip the initial number of units (for example, 100) and then take a unit (for example, 20 to 1) across a certain number of cells. So the sample to the unit, is approximated by the joint distribution.

Gibbs sample can obtain a posterior distribution sample of structural equation parameters under the condition of a priori distribution of covariance data and parameters. The point estimates, interval estimates, and standard errors of the parameters can be calculated using these sample data.

Gibbs sampling is a method of generating Markov chain, which can be used for Monte Carlo simulation to obtain a more complex multivariate distribution.

The specific method of Gibbs sampling: Suppose there is a k-dimensional random vector, now want to construct a k-dimensional vector with n samples (n sample Markov sequence), then (randomly) initialize a k-dimensional vector, and then fix the vector of the k-1 elements, extract the remaining element (generate a given posteriori random number), In this way, the whole vector is updated again, that is, a new sample is generated, and the whole is repeated n times to get a Markov chain.

Second: what you think

(1) Always, feel a sense of guilt, oneself now hard study of the C+stl family junior has all achieved their own; I now learn the AKI Chinese word breaker tool, other people's curriculum design is it; I have htmlparser is the curriculum design of the year, the emotional analysis is also the curriculum design, All sorts of classification clusters are the nutch of others, based on the search of the Hadoop family; am I making up for the year I spent on my postgraduate exams?

(2) What should graduate students do? Can't you just have one more card paper? Is China's education constitution a problem? Graduate students have to do research, learn the systematic analysis of the problem, rather than simply the mechanical writing a few lines of code, but broaden their horizons, learn a few model principles and their applications, learn the core ideas of those models.

(3) This may become very selfish, if so what have you brought to the tutor? What did you bring to the lab? The people who ask this question must be short-sighted people, in the long run, such people to the school to bring greater benefits to the country, of course, not directly to the teacher's purse drum up, because he did not have to the mentor's siege wrote a line of code, did not design a document

(4) If the tutor loses, is that OK? Because the teacher I am the exercise of power, against me, can not bring benefits to me, this certainly cannot, can not let him graduate, although he will bring greater benefits and contributions to the school country, but it has a hair relationship with me!

The way to Big data processing (Gibbs sampling)