Academic

Standard Bayesian methods typically scale poorly with the sample size, especially in complex, high-dimensional models. A popular approach to speed up the otherwise slow computation is using variational approximation of the posterior. Variational Bayes (VB) is frequently used in practice, however, so far it was considered as a black box procedure. Theoretical results started to emerge only in the last 1-2 years, but we still have a rather limited fundamental understanding about the limitations and guarantees of this procedure.  In this paper we provide general contraction guarantees for the VB method.  We apply these abstract results in context of high-dimensional linear and logistic regression models using spike-and-slab priors and in Gaussian process regression. We derive rate optimal contraction rates in both examples.