pyro-ppl / numpyro / 22284434899

93%

push

github

Add PSIS k-hat diagnostic for variational inference (#2139)

Implement Pareto Smoothed Importance Sampling (PSIS) diagnostic to
evaluate variational approximation quality, as requested in #1804.

The k-hat statistic is the shape parameter of a Generalized Pareto
Distribution fitted to the upper tail of importance weights. It
indicates whether the guide is a reliable approximation:
  k < 0.5: good (finite variance)
  0.5 <= k < 0.7: marginal (finite mean)
  k >= 0.7: unreliable

GPD fitting uses Zhang & Stephens (2009) with prior regularization
from Vehtari et al. (2024), matching Pyro's implementation and
Vehtari's reference code to ~1e-15.

93 of 93 new or added lines in 2 files covered. (100.0%)

14148 of 15272 relevant lines covered (92.64%)

1.69 hits per line