Machine Learning note Bayesian Learning (top)

Machine learning Notes (i)

Today formally began the study of machine learning, in order to motivate themselves to learn, but also to share ideas, decided to send their own experience of learning to the Internet to let everyone share.

Bayesian learning

Let's start with an example from the famous MLPP, a doctoral dissertation from Josh Tenenbaum, called a digital game.

In my own words: In order to decide who washes the dishes, xiaoming and his wife decide to play a game. Xiao Ming's wife first determines the properties of a number C, such as a prime or a mantissa of 3, and then gives a series of instances of such numbers in 1 to 100 d= {x1,..., XN}; finally give any number x please xiaoming to predict if X is in D. If Xiaoming is wrong, he will wash the dishes, of course, if you guessed the right to eat in the restaurant.

For example, when d={16, 8, 2, 64}, Xiaoming would guess that the initiator gave the C is 2 of the n-th square or even the form, so if x=32, Xiaoming will be very sure that the answer should be yes. Yes, but if X is 10, xiaoming may be a little hesitant. Fortunately, Xiao Ming is a yard farmer, Bayesian learning algorithm gives Xiaoming a tool for judging. Its basic idea is that the final probability (posterior probability) is proportional to the likelihood probability (likelihood) and the prior probability (prior) product

(1) Likelihood probability

In this example, assuming uniform sampling, it is obvious that the likelihood probability is given by the following formula:

That is, the larger the sample space, the less the probability of taking a particular set.

(2) Prior probability

A priori probability represents a supplement to likelihood probability. He can be drawn from historical data, or by experience. In this example, if D = {16,8,2,64}, then xiaoming can get two possible H. One is 2 n times, and the other is 2 N to remove 32. If only the likelihood probability is considered, the probability of the latter case is significantly greater, but from the experience of life we can know that unless the wife is very sick, it is unlikely that the second kind of morally bankrupt collection. So we give the "normal" set a relatively large priori probability, "abnormal" on the contrary, making the final result more consistent with our experience.

(3) post-test probability

The basic Bayesian formula is believed to be clear to everyone. When used because the denominator is a fixed value, so long as the numerator can be determined. The product of a priori probability and likelihood probability

is a concrete display.

It can be seen that even though the likelihood probability of the second hypothesis is greater, the posteriori probability is the first, which is in line with the experience of the people.

(Sadly, the result of this story is that the latter hypothesis is correct, and xiaoming is happy to wash the dishes again)

The addition of a priori probability allows us to deal with the occurrence of the "Black Swan event", and the specific mathematical explanations will be elaborated in the next article.

Machine Learning note Bayesian Learning (UP)