Skip to main content

Variational Inference via χ Upper Bound Minimization

Author(s): Dieng, Adji B; Tran, Dustin; Ranganath, Rajesh; Paisley, John; Blei, David M

To refer to this page use:
Full metadata record
DC FieldValueLanguage
dc.contributor.authorDieng, Adji B-
dc.contributor.authorTran, Dustin-
dc.contributor.authorRanganath, Rajesh-
dc.contributor.authorPaisley, John-
dc.contributor.authorBlei, David M-
dc.identifier.citationDieng, Adji Bousso, Dustin Tran, Rajesh Ranganath, John Paisley, and David Blei. "Variational Inference via χ Upper Bound Minimization." Advances in Neural Information Processing Systems 30 (2017), pp. 2732-2741.en_US
dc.description.abstractVariational inference (VI) is widely used as an efficient alternative to Markov chain Monte Carlo. It posits a family of approximating distributions q and finds the closest member to the exact posterior p. Closeness is usually measured via a divergence D(q||p) from q to p. While successful, this approach also has problems. Notably, it typically leads to underestimation of the posterior variance. In this paper we propose CHIVI, a black-box variational inference algorithm that minimizes Dχ(p||q), the χ-divergence from p to q. CHIVI minimizes an upper bound of the model evidence, which we term the χ upper bound (CUBO). Minimizing the CUBO leads to improved posterior uncertainty, and it can also be used with the classical VI lower bound (ELBO) to provide a sandwich estimate of the model evidence. We study CHIVI on three models: probit regression, Gaussian process classification, and a Cox process model of basketball plays. When compared to expectation propagation and classical VI, CHIVI produces better error rates and more accurate estimates of posterior variance.en_US
dc.format.extent2732 - 2741en_US
dc.relation.ispartofAdvances in Neural Information Processing Systemsen_US
dc.rightsAuthor's manuscripten_US
dc.titleVariational Inference via χ Upper Bound Minimizationen_US
dc.typeConference Articleen_US

Files in This Item:
File Description SizeFormat 
VariationalInferenceChiUppoerBoundMin.pdf2.19 MBAdobe PDFView/Download

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.