The beauty of Mathematics: an ordinary and magical Bayesian method--abstract

Original

Example1:

Give the following picture: (from Mackey's book)

Q: How many boxes are there on the back of a tree?

In fact, the answer must be a lot of, one, two, and even n boxes are possible (for example, there are rows of boxes in the back, lined up in a straight line), we can only see the first one:

However, the most correct, but also the most reasonable explanation, is a box, because if the tree behind the two or even more boxes, why from the big tree front look up, both sides of the same height, color is the same, this is not too coincidental. If our model is based on this picture, tell us that there are probably two boxes behind the tree, so the generalization ability of this model is not too bad.

Reasons for not overfitting:

the observed data will always have a variety of errors, such as observation errors (such as when you observe a MM after you are not careful, hand shake is an error occurred), so if you go too far to find a perfect interpretation of the observation data model, will fall into the so-called data over-Provisioning (overfitting) situation, an over-matching model tries to explain the error (noise) (and in fact the noise does not need to be explained), and obviously it is too far off.

Another reason for over-matching is that when observations are not "inaccurate" because of errors, but because there are too many factors in the real world to contribute to the results of the data, and unlike noise, these deviations are the result of a collective contribution by some other factor, not what your model can explain--noise that doesn't need to be explained-- A realistic model often extracts only a few factors that are highly correlated with the results (cause)

The beauty of Mathematics: an ordinary and magical Bayesian method--abstract