Broken stick Construction Process-Polya-Chinese Restaurant process

Introduction of three methods of constructing Dirichlet process

1. Broken stick Construction Process (stick-breaking construction)

This paper presents an explicit sampling method, which constructs a well-defined g, so that G satisfies the Dirichlet process.

Construction: The first step, given a positive real number, is first constructed from the beta distribution, where k is from 1 to, and then constructs a. The reason for this is that we want to get a probability mass function.

The second step is to sample a parameter sequence from a base distribution h in the parameter space, which is subject to the distribution H.

The third step is to put them together to form a discrete distribution, which is a sampling of the Dirichlet process.

Recorded as ~gem ().

1. Polya Weng Programme

This method does not explicitly construct the distribution G, but is based on the properties of the posterior distribution.

Methods: When n observations are observed from the Dirichlet process, these values may have k different values, and the next observed condition is distributed as follows:

, notice that only the desired part of the distribution is taken, which is the number of values.

This thing has an image of the understanding: we want to from a Weng and a distribution H to take the ball, the probability of the game from Weng is proportional to the number of the ball in Weng, the probability of taking the ball from H is proportional to. At first the Weng was empty, taking the ball from H and putting it in the Weng. If the ball is removed from the Weng, it is placed in a ball of the same color, so that the probability of each ball being taken out is proportional to the color of the ball already in the Weng.

1. Chinese Restaurant procedure (Chinese Restaurant process)

If we sample the Polya urn scheme from the Dirichlet process, they take K < n different values, then the N samples form K clusters. In other words, randomly sampling N observations according to the Polya URN scheme corresponds to a division of the integer set {1,..., N}, each of which has a certain probability that the distribution of this division is called the Chinese restaurant process.

To make the distinction more obvious, we write the category label, that is. Then there is.

The Chinese restaurant process is a clustering process, assuming that there are no customers in the restaurant, the first person who has just come in is randomly choosing a table to sit on, each table represents a class, and the backward customer chooses the table according to the following principles: choose the table with the probability of K, and choose a table with no one to sit down with probability. The more tables there are, the more likely it is to gather more customers to form a cluster effect.

The Chinese restaurant process has a nature to be used in the subsequent discussion--the commutative (exchangeability). It is said that the formation division if the same, then the sampling order is irrelevant, that is, after the formation of a clustering effect, regardless of the customer into the restaurant order, the probability of such clustering is the same.

Advantage : Because of the probability of introducing new categories in the classification, the number of clusters of such clusters does not need to be specified.

Broken stick Construction Process-Polya-Chinese Restaurant process