Deep Learning: October 18 (about random sampling)

 

Recently, looking at the RBMS network in deep learning, RBMS itself has a variety of formulas that are hard to understand. It is even more difficult to come up with a few samples of the first three. So I think we should take a look at the theoretical knowledge of James sampling first. After investigation, it is found that a type of James is random sampling. Therefore, this section also focuses on understanding random knowledge at a simple level. The reference knowledge is the basic idea of blog random simulation and the common sampling method (sampling). This blog post is the most common explanation found on the Internet. In fact, it is best for scholars to introduce various mathematical formulas in plain language, which is extremely important for beginners. Of course, I also refer.

Sampling refers to the probability distribution function of a sample X (in most cases multi-dimensional). This function is used to generate multiple sample points. Some may ask, what is the difficulty? Do matlaab and other tools have commands to generate samples of various distributions? For example, mean distribution. Yes, it is indeed true, but the generation of these distributed sample points is not very easy and needs to be carefully designed. It would be effort-consuming to design the sample generation method for every function in the function field. So there is a random sampling method, as long as it can approximate the theoretical result value is OK. Of course, this is only one reason for the random adoption of methods. It is purely personal understanding, and there are certainly many other factors.

The following sections describe common random sampling methods:

I. Reject -- accept sampling

This method uses a distribution that we can easily sample to simulate the distribution that needs to be sampled. It must meet the following conditions:

The specific collection process is as follows:

The Geometric Explanation is as follows:

The above explanation shows that a sample X is given, and then a Y value is randomly selected. The Y value is randomly generated under the MQ (x) contour, if the Y value falls between two curves, it is rejected. Otherwise, it is accepted. This is easy to understand. The derivation of the theory is free of charge. It's too boring, haha.

 

Ii. Sampling of importance.

My understanding of importance sampling is that the purpose of this method is not to generate a sample, but to find the definite points of a function, this is only because the method for determining the points is obtained by randomly using another one, which is easy to collect and distribute (my research is relatively simple and I can only understand this for the time being ). As shown in:

A large number of samples X are obtained through random sampling of Q (x), then the mean of f (x) * w (X) is obtained, and the integral I value is obtained. W (x) is the importance. If Q (x) has a high probability, the number of samples X is more, and the value of w (X) is more, it also indirectly reflects the more important it is.

3. Metropolis-Hasting

This method uses a suggested distribution to update samples with a certain probability, which is similar to reject-sampling is accepted. The process is as follows:

 

Iv. Gaussian sampling

Gibss requires that you know the conditional probability of an attribute in the sample under all other attributes, and then use this conditional probability to generate the sample values of each attribute. The process is as follows:

 

References:

Basic Idea of Random Simulation and common sampling methods (sampling)

Http://www.jdl.ac.cn/user/lyqing/StatLearning/StatlLearning_handout.html