Topic Modeling Papers and Tools

Papers

David M. Blei. Probabilistic Topic Models. Communications of the ACM, 55(4):77–84, 2012.
- A high-level overview of probabilistic topic models.
Adler J Perotte, Frank Wood, Noémie Elhadad, and Nicholas Bartlett. Hierarchically Supervised Latent Dirichlet Allocation. Neural Information Processing Systems, 2011.
The model HSLDA was applied to large-scale data from clinical document labeling and retail product categorization tasks.
D. Blei, T. Griffiths, and M. Jordan. The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. Journal of the ACM, 57:2 1–30, 2010.
David Newman1, Sarvnaz Karimi, and Lawrence Cavedon. Topic Models to Interpret MeSH – MEDLINE’s Medical Subject Headings. Advances in Artificial Intelligence, 2009.
The paper shows how topic models are useful for interpreting and understanding MeSH, the Medical Subject Headings applied to articles in MEDLINE.
C. Wang and D. Blei. Variational inference for the nested Chinese restaurant process. Neural Information Processing Systems, 2009.
D. Blei, J. McAuliffe. Supervised Topic Models. Neural Information Processing Systems 21, 2007.
D. Blei and J. Lafferty. Dynamic Topic Models. In Proceedings of the 23rd International Conference on Machine Learning, 2006.
Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101[476]:1566-1581, 2006.
David M. Blei, Andrew Ng, Michael Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022, 2003.

Implementations

Note: The table format and content of the first two rows are borrowed from David Blei's Topic Modeling page

Link	Model/Algorithm	Language	Author	Notes
lda-c	Latent Dirichlet allocation	C	D. Blei	This implements variational inference for LDA.
class-slda	Supervised topic models for classifiation	C++	C. Wang	Implements supervised topic models with a categorical response.
GibbsLDA++	A C/C++ Implementation of Latent Dirichlet Allocation	C/C++	Xuan-Hieu Phan and Cam-Tu Nguyen	Uses Gibbs Sampling technique for parameter estimation and inference. It is very fast and is designed to analyze hidden/latent topic structures of large-scale datasets including large collections of text/Web documents.
MALLET	A Machine Learning for Language Toolkit	Java	Andrew Kachites McCallum	Implements Gibbs sampling for LDA in Java using fast sampling methods. MALLET also includes support for data preprocessing, classification, and sequence tagging.
Gensim	A Python package for topic modelling.	Python	Radim Řehůřek	Includes distributed and online implementation of variational LDA.
Multithreaded LDA	Multithreaded extension of Blei's LDA implementation.	C	Ramesh Nallapati	Speeds up the computation by orders of magnitude depending on the number of processors.
Stanford Topic Modeling Toolbox	Scala implementation of LDA and Labeled LDA.	Scala	Daniel Ramage and Evan Rosen	Import and manipulate text from cells in Excel and other spreadsheets. Generate rich Excel-compatible outputs for tracking word usage across topics, time, and other groupings of data.

Corpora

200 topics from Wikipedia (240Mb, 71,986 docs)
200 topics from Ohsumed (a subset of MEDLINE abstracts, 156Mb, 233,442 abstracts)