We study the convergence properties of the Gibbs Sampler in the context of
posterior distributions arising from Bayesian analysis of Gaussian hierarchical
models. We consider centred and non-centred parameterizations as well as their
hybrids including the full family of partially non-centred parameterizations.
We develop a novel methodology based on multi-grid decompositions to derive
analytic expressions for the convergence rates of the algorithm for an
arbitrary number of layers in the hierarchy, while previous work was typically
limited to the two-level case. Our work gives a complete understanding for the
three-level symmetric case and this gives rise to approximations for the
non-symmetric case. We also give analogous, if less explicit, results for
models of arbitrary level. This theory gives rise to simple and
easy-to-implement guidelines for the practical implementation of Gibbs samplers
on conditionally Gaussian hierarchical models.