Aki Vehtari’s case studies
Cross-validation
Uncertainty in Bayesian leave-one-out cross-validation based model comparison
When cross-validation is not needed
- Simple model we trust - betablockers
When cross-validation is useful
- We don’t trust the model - roaches
- Complex model with posterior dependencies - collinear
- Avoiding double use of data in posterior predictive checking - Nabiximols
Comparison of discrete and continuos observation model
Integrated PSIS-LOO for varying intercept models: roaches
Cross-validation for hierarchical models: rats K-fold-CV
-
- Bayesian Stacking and Pseudo-BMA weights using the loo package
- Leave-one-out cross-validation for non-factorizable models
- Approximate leave-future-out cross-validation for time series models
- Using Leave-one-out cross-validation for large data
- Avoiding model refits in leave-one-out cross-validation with moment matching
Projection predictive model selection – projpred
- Student projpred variable selection workflow
- mesquite collinearity in variable selection
- bodyfat collinearity in variable selection
- candy random data vs original data variable selection
- winequality-red ordinal model variable selection
- See also projpred quick start vignette
Bayesian workflow
- Nabiximols: model checking and comparison, prior sensitivity analysis, model refinement
- Birthdays: iterative model building, use of fast approximate inference
- Friends: model checking and comprison before model interpretation.
- Disc golf putting: geometrical model for disc golf putting.
Gaussian processes
- Birthdays: Workflow demo for building time series model from many Gaussian process components
- Motorcycle: Gaussian process demo with heteroscedastic noise model and Hilbert basis function and covariance matrix implementations. In addition demonstrates the benefits of integration in Bayesian inference.
MCMC diagnostics and accuracy
- ESS comparison: Comparison of MCMC effective sample size (and MCSE) estimators
- Digits: How many digits to report and how many iterations to run
- Pareto-\hat{k} diagnostics:
Laplace approximation and Jacobian transformation
- Jacobian: Illustration of Laplace approximation and Jacobian adjustment in Stan.