Bayesian Nonnegative Matrix Factorization with Stochastic Variational Inference

David M. Blei Departments of Statistics and Computer Science, Columbia University, New York, NY 10027, USA

Michael I. Jordan Computer Science Division and Department of Statistics, University of California, Berkeley, CA 94720, USA

We present stochastic variational inference algorithms for two Bayesian nonnegative matrix factorization (NMF) models. These algorithms allow for fast processing of massive datasets. In particular, we derive stochastic algorithms for a Bayesian extension of the NMF algorithm of Lee and Seung (2001), and a matrix factorization model called correlated NMF, which is motivated by the correlated topic model (Blei and Lafferty, 2007). We apply our algorithms to roughly 1.8 million documents from the New York Times, comparing with online LDA (Hoffman et al., 2010b).