# Calibration of p-values from t-tests Transforms one- and two-sided p-values from t-tests to sample-size adjusted lower bounds on the Bayes factor for the point null hypothesis against the alternative. ```r tCalibrate(p, n, type="two.sided", alternative="normal", transform="id") ``` ## Arguments - `p`: a vector of p-values - `n`: a vector of sample sizes - `type`: either `"one.sided"` or `"two.sided"`. Defaults to `"two.sided"`. Specifies if the t-test (and the corresponding p-value) is one-sided or two-sided. - `alternative`: either `"simple"` or `"normal"`. Defaults to `"normal"`. Specifies the alternative hypotheses for the mean to consider. See Details for more information. - `transform`: either `"id"`, `"log"`, `"log2"` or `"log10"`. Defaults to `"id"`. Specifies how to transform the lower bound on the Bayes factor. `"id"` corresponds to no transformation. `"log"` refers to the natural logarithm, `"log2"` to the logarithm to the base 2 and `"log10"` to the logarithm to the base 10. ## Details For one-sided t-tests (`type="one.sided"`), `alternative="simple"` considers all simple point alternatives for the mean and `alternative="normal"` all shifted and possibly mirrored (if `direction=less`) half-normal distributions with mode at the null value. For two-sided t-tests (`type="two.sided"`), `alternative="simple"` considers all two-point distributions on the mean which are symmetric with respect to the null value and `alternative="normal"` all local normal distributions centered around the null value. Note that for `alternative="normal"`, the lower bound on the Bayes factor is the same for one-sided and two-sided p-values. For `type="one.sided", alternative="simple"`, the calibration is available in closed form and is given in Held & Ott (2018), equation (19). For `type="two.sided", alternative="simple"`, the calibration needs to be computed by numerical optimization and is given in Held & Ott (2018), equations (17) and (18). For `alternative="normal"`, the corresponding lower bound on the Bayes factor has been derived in Johnson (2005) and the calibration is proposed in Held & Ott (2016), Section 3. If `alternative="normal"`, `tCalibrate()` is a special case of `FCalibrate()`, i.e. `tCalibrate(p, n, alternative="normal")` is equivalent to `FCalibrate(p, n, d=1, alternative="chi.squared")`. One can show that if one considers the class of all alternatives which are symmetric with respect to the null value, one obtains the same lower bound as for the subclass of all symmetric two-point distributions. ## Returns A matrix containing the lower bounds on the Bayes factors as entries, for all combinations of p-value and sample size. The values for the k-th sample size (k-th entry in the vector `n`) and the specified p-values are given in the k-th row. ## References Held, L. and Ott, M. (2018). On p-values and Bayes factors. **Annual Review of Statistics and Its Application**, 5 , 393--419. Held, L. and Ott, M. (2016). How the maximal evidence of P-values against point null hypotheses depends on sample size. **The American Statistician**, 70 , 335--341. Johnson, V. E. (2005). Bayes factors based on test statistics. **Journal of the Royal Statistical Society, Series B** 67 :689--701. ## See Also `FCalibrate` ## Examples ```r tCalibrate(p=c(0.05, 0.01, 0.005), n=10, type="one.sided", alternative="simple") # two-sided alternatives tCalibrate(p=c(0.05, 0.01, 0.005), n=20, alternative="simple") tCalibrate(p=c(0.05, 0.01, 0.005), n=c(10, 20, 50)) # for two-sided alternatives, plot the minimum Bayes factors # as a function of the p-value par(mfrow=c(1,2), pty="s", las=1) # plot for simple alternatives p <- exp(seq(log(0.0001), log(0.3), by=0.01)) n <- c(10, 20, 50) minBF <- tCalibrate(p, n, alternative = "simple") # compare to the bound for large n bound1 <- zCalibrate(p, alternative="simple") matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", ylab="Min. Bayes factor", log="xy", xlab="two-sided t-test p-value", lty=1, lwd=2, axes=FALSE, main="Simple alternative") lines(p, bound1, col="gray", lty=2, lwd=2) my.at1 <- c(0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3) my.values1 <- as.character(c(format(c(0.0001,0.0003), nsmall=4, digits=4, scientific=FALSE), c(0.001, 0.003, 0.01, 0.03, 0.1, 0.3))) axis(1, at=my.at1, labels=my.values1) my.values2 <- c(3000, 1000, 300, 100, 30, 10, 3, 1) my.at2 <- 1/my.values2 my.ylegend2 <- c(paste("1/", my.values2[-length(my.values2)], sep=""), "1") axis(2, at=my.at2, labels=my.ylegend2) box() legend("bottomright", legend=rev(c("n=10", "n=20", "n=50", "n large")), lty=rev(c(rep(1,3), 2)), lwd=2, col=rev(c(1:3, "gray"))) # plot for local normal alternatives p <- exp(seq(log(0.0001), log(0.3), by=0.01)) n <- c(10, 20, 50) minBF <- tCalibrate(p, n) bound2 <- zCalibrate(p) matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", ylab="Min. Bayes factor", log="xy", xlab="two-sided t-test p-value", lty=1, lwd=2, axes=FALSE, main="Local normal alternatives") lines(p, bound2, col="gray", lty=2, lwd=2) axis(1, at=my.at1, labels=my.values1) axis(2, at=my.at2, my.ylegend2) box() legend("bottomright", legend=rev(c("n=10", "n=20", "n=50", "n large")), lty=rev(c(rep(1,3), 2)), lwd=2, col=rev(c(1:3, "gray"))) ```

tCalibrate function