Gelman-Rubin statistic

The Gelman-Rubin statistic allows a statement about the convergence of Monte Carlo simulations.

Definition

$J$ Monte Carlo simulations (chains) are started with different initial values. The samples from the respective burn-in phases are discarded. From the samples $x_{1}^{(j)},\dots ,x_{L}^{(j)}$ (of the j-th simulation), the variance between the chains and the variance in the chains is estimated:

${\overline {x}}_{j}={\frac {1}{L}}\sum _{i=1}^{L}x_{i}^{(j)}$ Mean value of chain j

${\overline {x}}_{*}={\frac {1}{J}}\sum _{j=1}^{J}{\overline {x}}_{j}$ Mean of the means of all chains

$B={\frac {L}{J-1}}\sum _{j=1}^{J}({\overline {x}}_{j}-{\overline {x}}_{*})^{2}$ Variance of the means of the chains

$W={\frac {1}{J}}\sum _{j=1}^{J}\left({\frac {1}{L-1}}\sum _{i=1}^{L}(x_{i}^{(j)}-{\overline {x}}_{j})^{2}\right)$ Averaged variances of the individual chains across all chains

An estimate of the Gelman-Rubin statistic $R$ then results as

$R={\frac {{\frac {L-1}{L}}W+{\frac {1}{L}}B}{W}}$.

When L tends to infinity and B tends to zero, R tends to 1.

A different formula is given by Vats & Knudson.

Alternatives

The Geweke Diagnostic compares whether the mean of the first x percent of a chain and the mean of the last y percent of a chain match.

Literature

• Vats, Dootika; Knudson, Christina (2021). "Revisiting the Gelman–Rubin Diagnostic". Statistical Science. 36 (4). arXiv:1812.09384. doi:10.1214/20-STS812.
• Gelman, Andrew; Rubin, Donald B. (1992). "Inference from Iterative Simulation Using Multiple Sequences". Statistical Science. 7 (4): 457–472. Bibcode:1992StaSc...7..457G. doi:10.1214/ss/1177011136. JSTOR 2246093.