prediction_summary_cv function

Cross-Validated Posterior Predictive Summaries

Given a set of observed data including a quantitative response variable y and an rstanreg model of y, this function returns 4 cross-validated measures of the model's posterior prediction quality: Median absolute prediction error (mae) measures the typical difference between the observed y values and their posterior predictive medians (stable = TRUE) or means (stable = FALSE). Scaled mae (mae_scaled) measures the typical number of absolute deviations (stable = TRUE) or standard deviations (stable = FALSE) that observed y values fall from their predictive medians (stable = TRUE) or means (stable = FALSE). within_50 and within_90 report the proportion of observed y values that fall within their posterior prediction intervals, the probability levels of which are set by the user. For hierarchical models of class lmerMod, the folds are comprised by collections of groups, not individual observations.

prediction_summary_cv(
  data,
  group,
  model,
  k,
  prob_inner = 0.5,
  prob_outer = 0.95
)

Arguments

• data: data frame including the variables in the model, both response y and predictors x
• group: a character string representing the name of the factor grouping variable, ie. random effect (only used for hierarchical models)
• model: an rstanreg model object with quantitative y
• k: the number of folds to use for cross validation
• prob_inner: posterior predictive interval probability (a value between 0 and 1)
• prob_outer: posterior predictive interval probability (a value between 0 and 1)

Returns

list

Examples

example_data <- data.frame(x = sample(1:100, 20))
example_data$y <- example_data$x*3 + rnorm(20, 0, 5)
example_model <- rstanarm::stan_glm(y ~ x,  data = example_data)
prediction_summary_cv(model = example_model, data = example_data, k = 2)