[Bayes] Bayesian School and frequency School

I,

"Detector, if I ask a statistician from the Bayesian school, if ......"
"[Throw] I am a neutron detector, not a maze guard. To be honest, is your mind broken ."
"[Throw]... yes"

The stalk of the maze guard:
There are two roads in the maze, respectively leading to the destination and traps. Each intersection has a guard, one of which only tells the truth and the other only tells lies, they all know what is behind the road and what each other is talking about. At this time, you can only choose one of them and ask a question, how to make sure you are on the right path.

Http://www.xkcd.com/1132/

========================================================== ==========

II,Frequency School (frequentists) Bayesian School (bayesians)

Laplace said: "probability theory only expresses common sense in mathematical formulas." Our mathematical models are just a Summary of the Laws of objective events. This is exactly what Bayesian theorem is.

Since Bayesian theorem is mentioned, we have to mention frequentists and bayesians ). the most important thing of the frequency School is the constant repetition (the more the better, the more infinitely). The Bayesian School focuses on sampling and distribution. although the rise of the Bayesian school was just over 20 years ago, since then, there has never been any debate between the two theoretical factions. here are some differences between the frequency School and Bayesian school.

• The frequency school holds that sampling is infinite. in the infinite sampling, the decision-making rules can be very accurate, while the Bayesian school believes that the world is changing all the time, and the unknown variables and events have a certain probability. This probability changes the state of the world at any time (the posterior probability mentioned above is the correction of the anterior probability ).
• The frequency school holds that the parameters of the model are fixed. After numerous samples of a model, all parameters should be the same, while the Bayesian school holds that the data should be fixed. our rules are derived from our observation and understanding of the world. what we see is true and true. parameters should be estimated from the observed things.
• The frequency school assumes that no model has a prior, and a prior has an important role in the Bayesian School.
• The frequency school advocates an evaluation paradigm. it has no prior and is more objective. the Bayesian school advocates a model method. create a model with unknown parameters. before the sample is observed, all parameters are uncertain. the observed sample value is used to estimate the parameters. the obtained parameters are imported into the model to make the current model best fit the observed data.

Https://blog-charliemorning.rhcloud.com/talk-about-navie-bayes/

Iii. Bayesian statistics

The key to Bayesian inference is that any inference must be based only on the posterior distribution, rather than the sample distribution.

Bayesian School and frequency school focus on prior distribution. Bayesian schools hold that the prior distribution can be subjective and does not require frequency interpretation. The frequency school holds that a prior distribution can be used in statistical inference only when the prior distribution is independent of subjective meaning and can be determined based on appropriate theories or past experience, otherwise, the objectivity will be lost.

First, the frequency theory first establishes an invalid model, and then computes the possibility of obtaining parameters from the actual data on the premise of this invalid model. If this possibility is small, we think that invalid models are not valid, so we can select alternative models. bayesian theory focuses on the probability that a model is established on the premise of the current data, and the specific probability value is obtained, this probability value is not used to judge a hypothesis.

Second, the frequency theory explains the probability: The frequency of an event that occurs for a long period of time; the Bayesian theory explains the probability to determine whether an event has occurred.

Thirdly, bayesian theory is good at utilizing previous knowledge and sampling data, while frequency theory only uses sampling data. Therefore, the posterior probability distribution obtained previously in Bayesian inference can be used as a prior probability of the next time.

Fourth, different interpretations of confidence intervals: In the frequency theory, 95% confidence intervals are interpreted as: 95 of the 100 confidence intervals calculated by 100 sampling contain the overall parameters, and 5 do not, it cannot be interpreted as 95% of the likelihood of a sample containing the overall parameter. This is because the overall parameters in classical statistics are treated as a constant value and cannot be interpreted from the perspective of probability. The confidence interval of bayesian theory can be interpreted as probability, in Bayesian analysis, the overall parameter is a random variable rather than a constant value.

The MCMC (Markov Chain Monte Carlo) method provides direct sampling from posterior distribution, which brings revolutionary breakthroughs in the practical application of Bayesian statistical methods.

Http://blog.sina.com.cn/s/blog_60864c1b0100dos3.html

========================================================== =====

Iv. Debate

Sender: dychdych (SIR), email area: Statistics
Question: frequentist and Bayesian
Mail site: unknown space-unnamed space (Fri Nov 14 19:48:02 2003) WWW-POST

Some people have learned many years of statistics and cannot tell the difference between the frequency School and the Bayesian school. What is subjective and objective? What is likelihood?
The posterior probability of a function, which is a phenomenon, is not the essence. The essential difference between the two is that the frequency school treats unknown parameters
The number is regarded as a common variable, and the sample is regarded as a random variable. The Bayesian school regards all variables as random variables.
The biggest difference between mathematics and statistics is that mathematics studies variables, while statistics studies random variables. Statistics
It is more natural to regard all variables as random variables.
If the Bayesian School is a pure statistician, the frequency School is a mathematical statistician, which is still in
The intermediate stage of the statistical transition, such as Xiao. Since you have changed from a fish to a frog, why keep your tail?
?
If all variables are random variables, many concepts of the frequency school become meaningless. For example, unbiased estimation
.
If E (t) = t, the statistic T is the unbiased estimator of the unknown parameter T. If the parameter T is a random variable
The equal sign is meaningless, because the expected E (t) of the statistic T is a quantity, and it cannot be equal to a random variable.
Trivial.
In addition, it is not as hard as the frequency school to explain the meaning of the confidence interval. What unknown parameters are unknown?
While the fixed value, while the interval is a random interval, because the endpoint of the interval is a statistic, it is also a random variable.
With the differences in observed samples, we get different range estimates. When the number of tests is large enough, there are about 95%
The interval contains the fixed unknown parameter. How troublesome! In order to be able to circle itself.
In history, the main reason why the Bayesian school has remained quiet is that the posterior probability to be calculated by the Bayesian school is very cumbersome.
Export, and many results are not explicitly displayed. Today with the high development of computers and various Monte Carlo Numerical Algorithms
The introduction and popularization of Bayesian school will eventually take the dominant place, and statistics at that time will be pure statistics.

Sender: yeren (savage), email area: Statistics
Title: Re: frequentist and Bayesian
Mail site: unknown space-unnamed space (Fri Nov 14 22:32:55 2003) WWW-POST

I don't agree with you.
Declare that I am also a Bayesian (or empirical Bayesian ).

The main difference between the frequency School and Bayesian school is whether prior probability distribution is allowed.
The frequency school does not regard all parameters as common variables (I think it should be known or unknown fixed
Variable, just like your term), such as hierarchical model and random effect model.
The Bayesian school also has common variables in the prior distribution, such as hyperprior parameter.

I disagree with your opinion on unbiased estimation, because your definition is unreasonable. If T is a random variable,
You can use E [T | T] = T, or get E [T] = m from the marginal distribution, a volume independent of T.

The advantage of Bayesian is that Bayesian inference is relatively simple, such as point estimation, interval estimation, and hypothesis test.
All can be obtained by posterior distribution, especially with the development of computer technology and the appearance of MCMC method.
It is possible to use and calculate the non-bounded posterior distribution. Moreover, its theoretical architecture is naturally in line with human progress.
. I just thought this morning that I could use the "always-in-the-air" method.
It is inappropriate to describe the Bayesian School.

However, the problem with Bayesian (full Bayesian) is that there is no prior information and it has been proved that it does not exist. All
Verification
After parameter transformation, it is inevitable to be subjective. However, the frequency school does not use the maximum likelihood estimation (MLE ).
Problem. The difficulty of the frequency school lies in how to use the previous experiences and pivot statistics.

Over the past few decades, the two schools have been arguing that each other will surely perish.
Signs. During this period, Bayesian, the compromise between the two, developed. Empirical Bayes and traditional bayesian
The difference is that it uses data to estimate (marginal maximum likelihood estimator, MMLE) a prior
Parameters in the distribution. Therefore, it is accepted by some frequency school scholars.

Apart from Bayesian and frequency, there are also likelihood schools. The likelihood school advocates using MLE and LR (likelihood
Ratio)
As the basis of inference, the widely used p-value is abolished. However, the likelihood school method is too difficult to apply, as though it is not currently available
What Dawn (I am too clear about the likelihood school, and you are welcome to refute it ).

Http://aimit.blog.edu.cn/2009/230160.html

========================================================== ==========

V. Discussion

Http://www.douban.com/group/topic/16719644/

Http://www.douban.com/group/topic/16951058/

To put it bluntly, the world view is different. The frequency school considers that the parameter exists objectively and does not change. Although unknown, it is a fixed value. The Bayesian school considers the parameter as a random value because it is not observed, there is no difference between it and a random number, so the parameters can also be distributed. I personally think this is consistent with some ideas of quantum mechanics.
To put it down, the frequency school is most concerned with the likelihood function, while the Bayesian school is most concerned with the posterior distribution. We will find that the posterior distribution is actually the likelihood function multiplied by the prior distribution and then
Normalize points to 1. Therefore, many methods of the two are the same.

Bayesian because all parameters are random variables and distributed, some sampling-based methods can be used.
(Such as MCMC) makes it easier for us to build complex models. The advantage of frequency school is that it does not assume a prior distribution, so it is more objective and unbiased. In some conservative fields (such as the pharmaceutical industry and law)
Bayesian methods are more trusted.