"SR" MAP

MAP: Maximum posteriori probability (Maximum a posteriori)

The estimation method obtains the point estimation of the hard-to-observe quantity based on empirical data. It is closely related to the Fisher method in maximum likelihood estimation,

But it uses an enlarged optimization target, which fuses the prior distributions of the estimators. Therefore, the maximum posteriori estimate can be regarded as the maximum likelihood estimation of the regularization .

"Reprinted from" Maximum Posterior assessment (MAP)-Cola ll-Blog Park https://www.cnblogs.com/liliu/archive/2010/11/24/1886110.html

The maximum posteriori estimate is a point estimate of the hard-to-observe quantity based on empirical data. Similar to the maximum likelihood estimate, but the greatest difference is that the maximum posteriori estimate incorporates a priori distribution of the estimated amount. Therefore, the maximum posteriori estimate can be regarded as the maximum likelihood estimate of the rule.

First, we review the maximum likelihood estimates in the previous article, assuming that X is an independent sample of the same distribution, θ is the model parameter, and F is the model we are using. Then the maximum likelihood estimate can be expressed as:

Now, suppose that the prior distribution of θ is g. With Bayesian theory, the posterior distribution of θ is shown in the following formula:

The goal of the final distribution is:

Note: The maximum posteriori estimate can be considered as a specific form of Bayesian estimation.

For example:

Suppose there are five bags, each with an unlimited amount of biscuits (cherry or lemon), and the ratio of the two flavors known to five bags is

Cherry 100%

Cherry 75% + Lemon 25%

Cherry 50% + Lemon 50%

Cherry 25% + Lemon 75%

Lemon 100%

If only the above conditions, the question from the same bag to get 2 of lemon biscuits, then this bag is most likely the above five which one?

We first use the maximum likelihood estimation to solve this problem and write out the likelihood function. Assuming that the probability of the lemon cookie being taken out of the bag is P (which we use to determine which bag is taken from), the likelihood function can be written

　　

Since the value of P is a discrete value, the 0,25%,50%,75%,1 described above. We just need to evaluate which value of these five values makes the likelihood function maximum and get the bag 5. Here is the result of the maximum likelihood estimate.

One problem with the above-mentioned maximum likelihood estimation is that the probability distribution of the model itself is not taken into account, and the problem of this cookie is extended below.

Suppose the probability of getting a bag 1 or 5 is 0.1, the probability of getting 2 or 4 is 0.2, the probability of getting 3 is 0.4, and the same answer to the above question? This is the time to change the map. We are based on the formula

Write our map function.

According to the description of test instructions, the values of P are 0,25%,50%,75%,1,g respectively 0.1,0.2,0.4,0.2,0.1. The results of the map function are as follows: 0,0.0125,0.125,0.28125,0.1. The result from the map estimate is the highest obtained from the fourth bag.

All of these are discrete variables, so what about continuous variables? When the hypothesis is independent of the same distribution, μ has a priori probability distribution of. Then we want to find the maximum posteriori probability of μ. According to the previous description, write the map function as:

At this point we take the logarithm on both sides. The maximum value of the above equation can be equal to min{}:

　　

The minimum value . The μ that can be obtained by derivation is:

The above is the process of map solving for continuous variables.

What we should note in the map is:

The biggest difference between map and MLE is that the probability distribution of the model parameter itself is added to the map, or. In Mle, the probability of the model parameter itself is uniform, that is, the probability is a fixed value.

"SR" MAP