compute_MSEv_eval_crit function

Mean Squared Error of the Contribution Function v(S)

Function that computes the Mean Squared Error (MSEv) of the contribution function v(s) as proposed by Frye et al. (2019) and used by Olsen et al. (2022).

compute_MSEv_eval_crit(
  internal,
  dt_vS,
  MSEv_uniform_comb_weights,
  MSEv_skip_empty_full_comb = TRUE
)

Arguments

• internal: List. Holds all parameters, data, functions and computed objects used within explain()

  The list contains one or more of the elements parameters, data, objects, iter_list, timing_list, main_timing_list, output, and iter_timing_list.

• dt_vS: Data.table of dimension n_coalitions times n_explain + 1 containing the contribution function estimates. The first column is assumed to be named id_coalition and containing the ids of the coalitions. The last row is assumed to be the full coalition, i.e., it contains the predicted responses for the observations which are to be explained.

• MSEv_uniform_comb_weights: Logical. If TRUE (default), then the function weights the coalitions uniformly when computing the MSEv criterion. If FALSE, then the function use the Shapley kernel weights to weight the coalitions when computing the MSEv criterion. Note that the Shapley kernel weights are replaced by the sampling frequency when not all coalitions are considered.

• MSEv_skip_empty_full_comb: Logical. If TRUE (default), we exclude the empty and grand coalitions when computing the MSEv evaluation criterion. This is reasonable as they are identical for all methods, i.e., their contribution function is independent of the used method as they are special cases not effected by the used method. If FALSE, we include the empty and grand coalitions. In this situation, we also recommend setting MSEv_uniform_comb_weights = TRUE, as otherwise the large weights for the empty and grand coalitions will outweigh all other coalitions and make the MSEv criterion uninformative.

Returns

List containing:

• MSEv: A data.table with the overall MSEv evaluation criterion averaged over both the coalitions and observations/explicands. The data.table

   also contains the standard deviation of the MSEv values for each explicand (only averaged over the coalitions) divided by the square root of the number of explicands.
  

• MSEv_explicand: A data.table with the mean squared error for each explicand, i.e., only averaged over the coalitions.

• MSEv_coalition: A data.table with the mean squared error for each coalition, i.e., only averaged over the explicands/observations. The data.table also contains the standard deviation of the MSEv values for each coalition divided by the square root of the number of explicands.

Details

The MSEv evaluation criterion does not rely on access to the true contribution functions nor the true Shapley values to be computed. A lower value indicates better approximations, however, the scale and magnitude of the MSEv criterion is not directly interpretable in regard to the precision of the final estimated Shapley values. Olsen et al. (2024)

illustrates in Figure 11 a fairly strong linear relationship between the MSEv criterion and the MAE between the estimated and true Shapley values in a simulation study. Note that explicands refer to the observations whose predictions we are to explain.

References

• c("\n", "Frye, C., de Mijolla, D., Begley, T., Cowton, L., Stanley, M., & Feige, I. (2021).\n", "Shapley explainability on the data manifold. In International Conference on Learning Representations.")
• c("\n", "Olsen, L. H., Glad, I. K., Jullum, M., & Aas, K. (2022). Using Shapley values and variational autoencoders to\n", "explain predictive models with dependent mixed features. Journal of machine learning research, 23(213), 1-51")
• c("\n", "Olsen, L. H. B., Glad, I. K., Jullum, M., & Aas, K. (2024). A comparative study of methods for estimating\n", "model-agnostic Shapley value explanations. Data Mining and Knowledge Discovery, 1-48")

Author(s)

Lars Henry Berge Olsen