An Infra-Structure for Performance Estimation of Predictive Models
Performance estimation using (e0 or .632) bootstrap
Class "Bootstrap"
CD diagrams for the post-hoc Boferroni-Dunn test
CD diagrams for the post-hoc Nemenyi test
Calculate some standard classification evaluation metrics of predictiv...
Class "ComparisonResults"
Class "CV"
Performance estimation using cross validation
Class "EstCommon"
Class "EstimationMethod"
Class "EstimationResults"
Obtain a set of descriptive statistics of the scores of a workflow on ...
Class "EstimationTask"
Obtaining the information returned by a workflow when applied to a tas...
Obtaining the predictions returned by a workflow when applied to a tas...
Obtaining the metric scores on the different iterations for a workflow...
Obtain the workflow object corresponding to an ID
Performance estimation using holdout and random resampling
Class "Holdout"
Check if a certain predictive task is a classification problem
Check if a certain predictive task is a regression problem
Fill in NA values with the values of the nearest neighbours
Class "LOOCV"
Performance estimation using Leave One Out Cross Validation
Performance estimation for time series prediction tasks using Monte Ca...
Merging several ComparisonResults class objects
The evaluation metrics estimated in an experiment
Obtains a summary of the individual metric scores obtained by each wor...
Class "MonteCarlo"
Statistical hypothesis testing on the observed paired differences in e...
Estimate the predictive performance of modeling alternatives on differ...
Class "PredTask"
Provide a ranking of workflows involved in an estimation process.
Calculate some standard regression evaluation metrics of predictive pe...
Obtain the target variable values of a prediction task
Obtains a dplyr data frame table object containing all the results of ...
Run a workflow on a predictive task
Obtains a list with the set of paired differences that are statistical...
SMOTE algorithm for unbalanced classification problems
A function for applying post-processing steps to the predictions of a ...
A function for applying data pre-processing steps
A function implementing a standard workflow for prediction tasks
Methods for Function subset in Package performanceEstimation
The prediction tasks involved in an estimation experiment
A function implementing sliding and growing window standard workflows ...
Obtain the workflow that best performed in terms of a metric on a task
Obtain the best scores from a performance estimation experiment
Class "Workflow"
The IDs of the workflows involved in an estimation experiment
Generate (parameter) variants of a workflow
An infra-structure for estimating the predictive performance of predictive models. In this context, it can also be used to compare and/or select among different alternative ways of solving one or more predictive tasks. The main goal of the package is to provide a generic infra-structure to estimate the values of different metrics of predictive performance using different estimation procedures. These estimation tasks can be applied to any solutions (workflows) to the predictive tasks. The package provides easy to use standard workflows that allow the usage of any available R modeling algorithm together with some pre-defined data pre-processing steps and also prediction post- processing methods. It also provides means for addressing issues related with the statistical significance of the observed differences.
Useful links