# LMG $R-squared$ decomposition for linear and logistic regression models `lmg` computes the Lindeman, Merenda and Gold (LMG) indices for correlated input relative importance by $R-squared$ decomposition for linear and logistic regression models. These indices allocates a share of $R-squared$ to each input based on the Shapley attribution system, in the case of dependent or correlated inputs. ```r lmg(X, y, logistic = FALSE, rank = FALSE, nboot = 0, conf = 0.95, max.iter = 1000, parl = NULL) ## S3 method for class 'lmg' print(x, ...) ## S3 method for class 'lmg' plot(x, ylim = c(0,1), ...) ``` ## Arguments - `X`: a matrix or data frame containing the observed covariates (i.e., features, input variables...). - `y`: a numeric vector containing the observed outcomes (i.e., dependent variable). If `logistic=TRUE`, can be a numeric vector of zeros and ones, or a logical vector, or a factor. - `logistic`: logical. If `TRUE`, the analysis is done via a logistic regression(binomial GLM). - `rank`: logical. If `TRUE`, the analysis is done on the ranks. - `nboot`: the number of bootstrap replicates for the computation of confidence intervals. - `conf`: the confidence level of the bootstrap confidence intervals. - `max.iter`: if `logistic=TRUE`, the maximum number of iterative optimization steps allowed for the logistic regression. Default is `1000`. - `parl`: number of cores on which to parallelize the computation. If `NULL`, then no parallelization is done. - `x`: the object returned by `lmg`. - `ylim`: the y-coordinate limits of the plot. - ``...``: arguments to be passed to methods, such as graphical parameters (see `par`). ## Returns `lmg` returns a list of class `"lmg"`, containing the following components: - **call**: the matched call. - **lmg**: a data frame containing the estimations of the LMG indices. - **R2s**: the estimations of the $R-squared$ for all possible sub-models. - **indices**: list of all subsets corresponding to the structure of R2s. - **w**: the Shapley weights. - **conf_int**: a matrix containing the estimations, biais and confidence intervals by bootstrap (if `nboot>0`). - **X**: the observed covariates. - **y**: the observed outcomes. - **logistic**: logical. `TRUE` if the analysis has been made by logistic regression. - **boot**: logical. `TRUE` if bootstrap estimates have been produced. - **nboot**: number of bootstrap replicates. - **rank**: logical. `TRUE` if a rank analysis has been made. - **parl**: number of chosen cores for the computation. - **conf**: level for the confidence intervals by bootstrap. ## Details The computation is done using the subset procedure, defined in Broto, Bachoc and Depecker (2020), that is computing all the $R-squared$ for all possible sub-models first, and then affecting the Shapley weights according to the Lindeman, Merenda and Gold (1980) definition. For logistic regression (`logistic=TRUE`), the $R-squared$ value is equal to: $$ R^2 = 1-\frac{\textrm{model deviance}}{\textrm{null deviance}}R-squared= 1 - (model deviance)/(null deviance) $$ If either a logistic regression model (`logistic = TRUE`), or any column of `X` is categorical (i.e., of class `factor`), then the rank-based indices cannot be computed. In both those cases, `rank = FALSE` is forced by default (with a `warning`). If too many cores for the machine are passed on to the `parl` argument, the chosen number of cores is defaulted to the available cores minus one. ## References Broto B., Bachoc F. and Depecker M. (2020) **Variance Reduction for Estimation of Shapley Effects and Adaptation to Unknown Input Distribution.** SIAM/ASA Journal on Uncertainty Quantification, 8(2). D.V. Budescu (1993). **Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression.** Psychological Bulletin, 114:542-551. L. Clouvel, B. Iooss, V. Chabridon, M. Il Idrissi and F. Robin, 2024, **An overview of variance-based importance measures in the linear regression context: comparative analyses and numerical tests**, Preprint. [https://hal.science/hal-04102053](https://hal.science/hal-04102053) U. Gromping (2006). **Relative importance for linear regression in R: the Package relaimpo.** Journal of Statistical Software, 17:1-27. M. Il Idrissi, V. Chabridon and B. Iooss (2021). **Developments and applications of Shapley effects to reliability-oriented sensitivity analysis with correlated inputs**, Environmental Modelling & Software, 143, 105115, 2021 M. Il Idrissi, V. Chabridon and B. Iooss (2021). **Mesures d'importance relative par decompositions de la performance de modeles de regression,** Actes des 52emes Journees de Statistiques de la Societe Francaise de Statistique (SFdS), pp 497-502, Nice, France, Juin 2021 B. Iooss, V. Chabridon and V. Thouvenot, **Variance-based importance measures for machine learning model interpretability**, Congres lambda-mu23, Saclay, France, 10-13 octobre 2022 [https://hal.science/hal-03741384](https://hal.science/hal-03741384) Lindeman RH, Merenda PF, Gold RZ (1980). **Introduction to Bivariate and Multivariate Analysis.** Scott, Foresman, Glenview, IL. ## Author(s) Marouane Il Idrissi ## Examples ```r library(parallel) library(doParallel) library(foreach) library(gtools) library(boot) library(mvtnorm) set.seed(1234) n <- 1000 beta<-c(1,-1,0.5) sigma<-matrix(c(1,0,0, 0,1,-0.8, 0,-0.8,1), nrow=3, ncol=3) ############################ # Gaussian correlated inputs X <-rmvnorm(n, rep(0,3), sigma) colnames(X)<-c("X1","X2", "X3") ############################# # Linear Model y <- X%*%beta + rnorm(n,0,2) # Without Bootstrap confidence intervals x<-lmg(X, y) print(x) plot(x) # With Boostrap confidence intervals x<-lmg(X, y, nboot=100, conf=0.95) print(x) plot(x) # Rank-based analysis x<-lmg(X, y, rank=TRUE, nboot=100, conf=0.95) print(x) plot(x) ############################ # Logistic Regression y<-as.numeric(X%*%beta + rnorm(n)>0) x<-lmg(X,y, logistic = TRUE) plot(x) print(x) # Parallel computing #x<-lmg(X,y, logistic = TRUE, parl=2) #plot(x) #print(x) ``` ## See Also `pcc`, `src`, `johnson`, `shapleyPermEx`, `shapleysobol_knn`, `pmvd`, `pme_knn`

lmg function