Dynamic Topic Modeling.pdf

The proposed model, Dynamic Correlated Topic Model (DCTM), extends Correlated Topic Models (CTM) to capture the evolution of topic correlation and word co-occurrence over time. DCTM uses Gaussian processes (GPs) to model the temporal dynamics of topic representations and generalized Wishart processes (GWPs) to model the evolution of topic correlations.

Key Components:

1. Temporal Prior Distribution: Derived from GPs, allowing for continuous-time modeling and interpolation/extrapolation of topic representations.
2. Dynamic Distribution for Mixtures of Topics: Extends CTM's prior distribution using GPs for the mean and GWPs for the covariance matrices.
3. Stochastic Variational Inference Method: Enables mini-batch training and scalability to millions of documents using a deep neural network to encode the variational posterior.

Contributions:

1. Full Dynamic Version of CTM: Models the evolution of topic representations, popularity, and correlations.
2. Scalable Variational Inference Method: Allows for efficient training on large datasets.

Related Work:

1. Static Topic Models: LDA, CTM, and variants with modifications to the prior distribution.
2. Dynamic Topic Models: Extensions of topic models to capture temporal changes, including latent Wiener processes and forward-backward learning algorithms.

Outline:

1. Introduction
2. Related Work
3. Generalized Dynamic Correlated Topic Model
4. Efficient Variational Inference Procedure
5. Experiments and Validation
6. Conclusion and Future Research Directions