A topic distribution is a probability distribution over the vocabulary that describes how a topic uses words. In LDA, topic k is represented by βₖ, shared across documents, while each document has topic proportions θᵈ. The model learns both from a term-document matrix A, which can be factorized as A = WH. section 9.1
Use the distribution to interpret and implement the topic model:
- Treat each βₖ as a vocabulary-sized vector whose entries express the topic's word usage. section 9.1
- Distinguish βₖ from θᵈ: βₖ describes a topic across documents, whereas θᵈ measures a document's mixture of topics. section 9.1
- Initialize each topic distribution from a Dirichlet distribution governed by η, add a small value to avoid zeros, and renormalize. section 9.1.1
- Infer distributions with mean-field variational Bayes, update corpus-level parameters using learning rate ρₜ, and inspect top-K words. section 9.1.1
Interpreting inferred topics
Applied to the 20 Newsgroups dataset, the inferred topic distributions can be visualized as word clouds; their top-K words align with recognizable document categories. section 9.1.1
Figure 9.1 Latent Dirichlet allocation graphical model
Inspect the graphical model to trace the relationship among words, topic labels, topic distributions, document proportions, and hyperparameters. section 9.1
