Nested Cross Validation for the Relaxed Lasso and Other Machine Learning Models
Identify model based upon AIC criteria from a stepreg() putput
Fit multiple Artificial Neural Network models on "tabular" provided as...
Fit an Artificial Neural Network model on "tabular" provided as a matr...
Get the best models for the steps of a stepreg() fit
Generate foldid's by 0/1 factor for bootstrap like samples where uniqu...
calculate cross-entry for multinomial outcomes
Construct calibration plots for a nested.glmnetr output object
Calculate the CoxPH saturated log-likelihood
Get a cross validation informed relaxed lasso model fit. Available to ...
Cross validation informed stepwise regression model fit.
Calculate deviance ratios for CV based
Output to console the elapsed and split times
Get elapsed time in c(hour, minute, secs)
Generate foldid's by factor levels
Get foldid's with branching for cox, binomial and gaussian models
Get foldid's when id variable is used to identify groups of dependent ...
Get seeds to store, facilitating replicable results
A redirect to nested.cis()
A redirect to nested.compare
Generate example data
Calculate performance measure "nominal" CI's and p's
Compare cross validation fit performances from a nested.glmnetr output...
Compare cross validation fit performances from a nested.glmnetr output...
Using (nested) cross validation, describe and compare some machine lea...
Fit a Random Forest model on data provided in matrix and vector format...
Plot nested cross validation performance summaries
Plot nested cross validation performance summaries
Plot cross-validation deviances, or model coefficients.
Plot the relaxed lasso coefficients.
Plot results from a nested.glmnetr() output
Get predicteds for an Artificial Neural Network model fit in nested.gl...
Give predicteds for elastic net models form a nested..glmnetr() output...
Give predicteds for elastic net models form a nested..glmnetr() output...
Give predicteds for elastic net models form a nested.glmnetr() output ...
Beta's or predicteds based upon a cv.stepreg() output object.
Give predicteds based upon the cv.glmnet output object contained in th...
A redirect to the summary() function for nested.glmnetr() output objec...
Print output from orf_tune() function
Print output from rf_tune() function
Rederive Oblique Random Forest models not kept in nested.glmnetr() out...
Rederive Random Forest models not kept in nested.glmnetr() output
Rederive XGB models not kept in nested.glmnetr() output
Fit a Random Forest model on data provided in matrix and vector format...
round elements of a summary.glmnetr() output
Fit the steps of a stepwise regression.
Output summary for elastic net models fit within a nested.glmnetr() ou...
Output summary for elastic net models fit within a nested.glmnetr() ou...
Summarize results from a cv.stepreg() output object.
Summarize a nested.glmnetr() output object
Summarize output from rf_tune() function
Summarize output from rf_tune() function
Briefly summarize steps in a stepreg() output object, i.e. a stepwise ...
Get a simple XGBoost model fit (no tuning)
Get a tuned XGBoost model fit
Cross validation informed Relaxed LASSO (or more generally elastic net), gradient boosting machine ('xgboost'), Random Forest ('RandomForestSRC'), Oblique Random Forest ('aorsf'), Artificial Neural Network (ANN), Recursive Partitioning ('RPART') or step wise regression models are fit. Cross validation leave out samples (leading to nested cross validation) or bootstrap out-of-bag samples are used to evaluate and compare performances between these models with results presented in tabular or graphical means. Calibration plots can also be generated, again based upon (outer nested) cross validation or bootstrap leave out (out of bag) samples. For some datasets, for example when the design matrix is not of full rank, 'glmnet' may have very long run times when fitting the relaxed lasso model, from our experience when fitting Cox models on data with many predictors and many patients, making it difficult to get solutions from either glmnet() or cv.glmnet(). This may be remedied by using the 'path=TRUE' option when calling glmnet() and cv.glmnet(). Within the 'glmnetr' package the approach of path=TRUE is taken by default. other packages doing similar include 'nestedcv' <https://cran.r-project.org/package=nestedcv>, 'glmnetSE' <https://cran.r-project.org/package=glmnetSE> which may provide different functionality when performing a nested CV. Use of the 'glmnetr' has many similarities to the 'glmnet' package and it could be helpful for the user of 'glmnetr' also become familiar with the 'glmnet' package <https://cran.r-project.org/package=glmnet>, with the "An Introduction to 'glmnet'" and "The Relaxed Lasso" being especially useful in this regard.