We study the estimation of the latent variable Gaussian graphical model
(LVGGM), where the precision matrix is the superposition of a sparse matrix and
a low-rank matrix. In order to speed up the estimation of the sparse plus
low-rank components, we propose a sparsity constrained maximum likelihood
estimator based on matrix factorization, and an efficient alternating gradient
descent algorithm with hard thresholding to solve it. Our algorithm is orders
of magnitude faster than the convex relaxation based methods for LVGGM. In
addition, we prove that our algorithm is guaranteed to linearly converge to the
unknown sparse and low-rank components up to the optimal statistical precision.
Experiments on both synthetic and genomic data demonstrate the superiority of
our algorithm over the state-of-the-art algorithms and corroborate our theory.