# Nonparametric Tests of Regression Coefficients Assuming a linear model of the form [REMOVE_ME]$$ Y = \alpha + X \beta + \epsilon [REMOVE_ME_2]$$ or [REMOVE_ME]$$ Y = \alpha + X \beta + Z \gamma + \epsilon [REMOVE_ME_2]$$ this function implements permutation tests of $H_0: \beta = \beta_0$ where $\beta_0$ is the user-specified null vector. ## Description Assuming a linear model of the form $$ Y = \alpha + X \beta + \epsilon $$ or $$ Y = \alpha + X \beta + Z \gamma + \epsilon $$ this function implements permutation tests of $H_0: \beta = \beta_0$ where $\beta_0$ is the user-specified null vector. ```r np.reg.test(x, y, z = NULL, method = NULL, beta = NULL, homosced = FALSE, lambda = 0, R = 9999, parallel = FALSE, cl = NULL, perm.dist = TRUE, na.rm = TRUE) ``` ## Arguments - `x`: Matrix of predictor variables (`n` by `p`). - `y`: Response vector or matrix (`n` by `m`). - `z`: Optional matrix of nuisance variables (`n` by `q`). - `method`: Permutation method. See Details. - `beta`: Null hypothesis value for $\beta$ (`p` by `m`). Defaults to matrix of zeros. - `homosced`: Are the $\epsilon$ terms homoscedastic? If `FALSE` (default), a robust Wald test statistic is used. Otherwise the classic $F$ test statistic is used. - `lambda`: Scalar or vector of ridge parameter(s). Defaults to vector of zeros. - `R`: Number of resamples for the permutation test (positive integer). - `parallel`: Logical indicating if the `parallel` package should be used for parallel computing (of the permutation distribution). Defaults to `FALSE`, which implements sequential computing. - `cl`: Cluster for parallel computing, which is used when `parallel = TRUE`. Note that if `parallel = TRUE` and `cl = NULL`, then the cluster is defined as `makeCluster(2L)` to use two cores. To make use of all available cores, use the code `cl = makeCluster(detectCores())`. - `perm.dist`: Logical indicating if the permutation distribution should be returned. - `na.rm`: If `TRUE` (default), the arguments `x` and `y` (and `z` if provided) are passed to the `na.omit` function to remove cases with missing data. ## Details With no nuisance variables in the model (i.e., `z = NULL`), there are three possible options for the `method` argument: |||| |:--|:--|:--| |Method|Model| |perm|$P Y = \alpha + X \beta + \epsilon$| |flip|$S Y = \alpha + X \beta + \epsilon$| |both|$P S Y = \alpha + X \beta + \epsilon$| where $P$ is a permutation matrix and $S$ is a sign-flipping matrix. With nuisance variables in the model, there are eight possible options for the `method` argument: |||| |:--|:--|:--| |Method|Name|Model| |HJ|Huh-Jhun|$P Q' R_z Y = \alpha + Q' R_z X \beta + \epsilon$| |KC|Kennedy-Cade|$P R_z Y = \alpha + R_z X \beta + \epsilon$| |SW|Still-White|$P R_z Y = \alpha + X \beta + \epsilon$| |TB|ter Braak|$(P R_m + H_m) Y = \alpha + X \beta + Z \gamma + \epsilon$| |FL|Freedman-Lane|$(P R_z + H_z) Y = \alpha + X \beta + Z \gamma + \epsilon$| |MA|Manly|$P Y = \alpha + X \beta + Z \gamma + \epsilon$| |OS|O'Gorman-Smith|$Y = \alpha + P R_z X \beta + Z \gamma + \epsilon$| |DS|Draper-Stoneman|$Y = \alpha + P X \beta + Z \gamma + \epsilon$| where $P$ is permutation matrix and $Q$ is defined as $R_z = Q Q'$ with $Q'Q = I$. Note that $H_z$ is the hat matrix for the nuisance variable design matrix, and $R_z = I - H_z$ is the corresponding residual forming matrix. Similarly, $H_m$ and $R_m$ are the hat and residual forming matrices for the full model including the predictor and nuisance variables. ## Returns - **statistic**: Test statistic value. - **p.value**: p-value for testing $H_0: \beta = \beta_0$. - **perm.dist**: Permutation distribution of `statistic`. - **method**: Permutation method. - **null.value**: Null hypothesis value for $\beta$. - **homosced**: Homoscedastic errors? - **lambda**: Ridge parameters. - **R**: Number of resamples. - **exact**: Exact permutation test? See Note. - **coefficients**: Least squares estimates of $\alpha$, $\beta$, and $\gamma$ (if applicable). - **univariate**: Univariate test statistic value for $j$-th variable (for multivariate inputs). - **adj.p.value**: Adjusted p-value for testing significance of $j$-th variable (for multivariate inputs). ## References DiCiccio, C. J., & Romano, J. P. (2017). Robust permutation tests for correlation and regression coefficients. Journal of the American Statistical Association, 112(519), 1211-1220. doi: 10.1080/01621459.2016.1202117 Draper, N. R., & Stoneman, D. M. (1966). Testing for the inclusion of variables in linear regression by a randomisation technique. Technometrics, 8(4), 695-699. doi: 10.2307/1266641 Freedman, D., & Lane, D. (1983). A nonstochastic interpretation of reported significance levels. Journal of Business and Economic Statistics, 1(4), 292-298. doi: 10.2307/1391660 Helwig, N. E. (2019a). Statistical nonparametric mapping: Multivariate permutation tests for location, correlation, and regression problems in neuroimaging. WIREs Computational Statistics, 11(2), e1457. doi: 10.1002/wics.1457 Helwig, N. E. (2019b). Robust nonparametric tests of general linear model coefficients: A comparison of permutation methods and test statistics. NeuroImage, 201, 116030. doi: 10.1016/j.neuroimage.2019.116030 Huh, M.-H., & Jhun, M. (2001). Random permutation testing in multiple linear regression. Communications in Statistics - Theory and Methods, 30(10), 2023-2032. doi: 10.1081/STA-100106060 Kennedy, P. E., & Cade, B. S. (1996). Randomization tests for multiple regression. Communications in Statistics - Simulation and Computation, 25(4), 923-936. doi: 10.1080/03610919608813350 Manly, B. (1986). Randomization and regression methods for testing for associations with geographical, environmental and biological distances between populations. Researches on Population Ecology, 28(2), 201-218. doi: 10.1007/BF02515450 Nichols, T. E., Ridgway, G. R., Webster, M. G., & Smith, S. M. (2008). GLM permutation: nonparametric inference for arbitrary general linear models. NeuroImage, 41(S1), S72. O'Gorman, T. W. (2005). The performance of randomization tests that use permutations of independent variables. Communications in Statistics - Simulation and Computation, 34(4), 895-908. doi: 10.1080/03610910500308230 Still, A. W., & White, A. P. (1981). The approximate randomization test as an alternative to the F test in analysis of variance. British Journal of Mathematical and Statistical Psychology, 34(2), 243-252. doi: 10.1111/j.2044-8317.1981.tb00634.x ter Braak, C. J. F. (1992). Permutation versus bootstrap significance tests in multiple regression and ANOVA. In K. H. Jockel, G. Rothe, & W. Sendler (Eds.), Bootstrapping and related techniques. Lecture notes in economics and mathematical systems, vol 376 (pp. 79-86). Springer. White, H. (1980). A heteroscedasticity-consistent covariance matrix and a direct test for heteroscedasticity. Econometrica, 48(4), 817-838. doi: 10.2307/1912934 Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., & Nichols, T. E. (2014). Permutation inference for the general linear model. NeuroImage, 92, 381-397. doi: 10.1016/j.neuroimage.2014.01.060 ## Author(s) Nathaniel E. Helwig ## Note If `method = "flip"`, the permutation test will be exact when the requested number of resamples `R` is greater than `2^n` minus one. In this case, the permutation distribution `perm.dist` contains all `2^n` possible values of the test statistic. If `method = "both"`, the permutation test will be exact when the requested number of resamples `R` is greater than `factorial(n) * (2^n)` minus one. In this case, the permutation distribution `perm.dist` contains all `factorial(n) * (2^n)` possible values of the test statistic. If `method = "HJ"`, the permutation test will be exact when the requested number of resamples `R` is greater than `factorial(n-q-1)` minus one. In this case, the permutation distribution `perm.dist` contains all `factorial(n-q-1)` possible values of the test statistic. Otherwise the permutation test will be exact when the requested number of resamples `R` is greater than `factorial(n)` minus one. In this case, the permutation distribution `perm.dist` contains all `factorial(n)` possible values of the test statistic. ## Multivariate Tests If the input `y` is a matrix with $m > 1$ columns, the multivariate test statistic is defined as `statistic = max(univariate)` given that the `univariate` test statistics are non-negative. The global null hypothesis (across all $m$ variables) is tested by comparing the observed `statistic` to the permutation distribution `perm.dist`. This produces the `p.value` for testing the global null hypothesis. The local null hypothesis (separately for each variable) is tested by comparing the `univariate` test statistic to `perm.dist`. This produces the adjusted p-values (`adj.p.values`), which control the familywise Type I error rate across the $m$ tests. ## See Also `plot.np.reg.test` S3 plotting method for visualizing the results ## Examples ```r ######******###### UNIVARIATE ######******###### ###***### TEST ALL COEFFICIENTS ###***### # generate data set.seed(1) n <- 10 x <- cbind(rnorm(n), rnorm(n)) y <- rnorm(n) # Wald test (method = "perm") set.seed(0) np.reg.test(x, y) # F test (method = "perm") set.seed(0) np.reg.test(x, y, homosced = TRUE) ###***### TEST SUBSET OF COEFFICIENTS ###***### # generate data set.seed(1) n <- 10 x <- rnorm(n) z <- rnorm(n) y <- 3 + 2 * z + rnorm(n) # Wald test (method = "HJ") set.seed(0) np.reg.test(x, y, z) # F test (method = "HJ") set.seed(0) np.reg.test(x, y, z, homosced = TRUE) ## Not run: ######******###### MULTIVARIATE ######******###### ###***### TEST ALL COEFFICIENTS ###***### # generate data set.seed(1) n <- 10 x <- cbind(rnorm(n), rnorm(n)) y <- matrix(rnorm(n * 3), nrow = n, ncol = 3) # multivariate Wald test (method = "perm") set.seed(0) np.reg.test(x, y) # multivariate F test (method = "perm") set.seed(0) np.reg.test(x, y, homosced = TRUE) ###***### TEST SUBSET OF COEFFICIENTS ###***### # generate data set.seed(1) n <- 10 x <- rnorm(n) z <- rnorm(n) y <- cbind(1 + 3 * z + rnorm(n), 2 + 2 * z + rnorm(n), 3 + 1 * z + rnorm(n)) # multivariate Wald test (method = "HJ") set.seed(0) np.reg.test(x, y, z) # multivariate F test (method = "HJ") set.seed(0) np.reg.test(x, y, z, homosced = TRUE) ## End(Not run) ```

np.reg.test function