LDA: Derivation of variation

Lda and latent diriclet allocation are the most basic Bayesian models. In this paper, we need to study lda's variational derivation method. It is of great significance.

I. symbol Definition

: The number of topics
? : The number of documents
? : The number of terms in vocabulary
? : Index topic
? : Index document
? : Index word
? : Denote a word

In LDA:
: Model Parameter
? : Model Parameter
?, : Hidden variables.

Graph Model:
Introduce variational parameter:
? : Dirichlet Parameter
? : Multinomial Parameter

We introduce variational distribution, a fully factorized Model

? Note that ,? Is the posterior distribution. We have removed given ??

Ii. Overview

We use variational EM algorithm:
In e step, we use variational approximation to posterior to optimize variational parameters and find the most reliable posterior distribution.
In M step, we promote lower bound with respect to the model parameters.

Specific algorithms:
E-step: for each document, find optimal values of the variational Parameters

? M-step: Maximize the lower bound with respect to the model parameters ?? And?

?

?

?

?

?

LDA: Derivation of variation