accuracy function

Minimum maximum accuracy, mean absolute percent error, median absolute error, root mean square error, coefficient of variation, and Efron's pseudo r-squared

Produces a table of fit statistics for multiple models.

accuracy(fits, plotit = FALSE, digits = 3, ...)

Arguments

• fits: A series of model object names. Must be a list of model objects or a single model object.
• plotit: If TRUE, produces plots of the predicted values vs. the actual values for each model.
• digits: The number of significant digits in the output.
• ...: Other arguments passed to plot.

Returns

A list of two objects: The series of model calls, and a data frame of statistics for each model.

Details

Produces a table of fit statistics for multiple models: minimum maximum accuracy, mean absolute percentage error, median absolute error, root mean square error, normalized root mean square error, Efron's pseudo r-squared, and coefficient of variation.

For minimum maximum accuracy, larger indicates a better fit, and a perfect fit is equal to 1.

For mean absolute error (MAE), smaller indicates a better fit, and a perfect fit is equal to 0. It has the same units as the dependent variable. Note that here, MAE is simply the mean of the absolute values of the differences of predicted values and the observed values (MAE = mean(abs(predy - actual))). There are other definitions of MAE and similar-sounding terms.

Median absolute error (MedAE) is similar, except employing the median rather than the mean.

For mean absolute percent error (MAPE), smaller indicates a better fit, and a perfect fit is equal to 0. The result is reported as a fraction. That is, a result of 0.1 is equal to 10 percent.

Root mean square error (RMSE) has the same units as the predicted values.

Normalized root mean square error (NRMSE) is RMSE divided by the mean or the median of the values of the dependent variable.

Efron's pseudo r-squared is calculated as 1 minus the residual sum of squares divided by the total sum of squares. For linear models (lm model objects), Efron's pseudo r-squared will be equal to r-squared. For other models, it should not be interpreted as r-squared, but can still be useful as a relative measure.

CV.prcnt is the coefficient of variation for the model. Here it is expressed as a percent. That is, a result of 10 = 10 percent.

Model objects currently supported: lm, glm, nls, betareg, gls, lme, lmer, lmerTest, glmmTMB, rq, loess, gam, glm.nb, glmRob, mblm, and rlm.

Examples

data(BrendonSmall)
BrendonSmall$Calories = as.numeric(BrendonSmall$Calories)
BrendonSmall$Calories2 = BrendonSmall$Calories ^ 2
model.1 = lm(Sodium ~ Calories, data = BrendonSmall)

accuracy(model.1, plotit=FALSE)

model.2 = lm(Sodium ~ Calories + Calories2, data = BrendonSmall)
model.3 = glm(Sodium ~ Calories, data = BrendonSmall, family="Gamma")
quadplat = function(x, a, b, clx) {
          ifelse(x  < clx, a + b * x   + (-0.5*b/clx) * x   * x,
                           a + b * clx + (-0.5*b/clx) * clx * clx)}
model.4 = nls(Sodium ~ quadplat(Calories, a, b, clx),
              data = BrendonSmall,
              start = list(a=519, b=0.359, clx = 2300))
              
accuracy(list(model.1, model.2, model.3, model.4), plotit=FALSE)

### Perfect and poor model fits
X = c(1, 2,  3,  4,  5,  6, 7, 8, 9, 10, 11, 12)
Y = c(1, 2,  3,  4,  5,  6, 7, 8, 9, 10, 11, 12)
Z = c(1, 12, 13, 6, 10, 13, 4, 3, 5,  6, 10, 14)
perfect = lm(Y ~ X)
poor    = lm(Z ~ X)
accuracy(list(perfect, poor), plotit=FALSE)

References

https://rcompanion.org/handbook/G_14.html

See Also

compareLM, compareGLM, nagelkerke

Author(s)

Salvatore Mangiafico, mangiafico@njaes.rutgers.edu