calibrate_me_CV_errors function

calibrate_me_CV_errors

trains and evaluates calibration models using n_seeds-times repeated folds-Cross-Validation (CV).model_idx specifies which models should be trained.

Model training and evaluation is repeated n_seeds-times with a different training/test set partition scheme for the CV each time.

calibrate_me_CV_errors(actual, predicted, model_idx, folds = 10, n_seeds,
  nCores)

Arguments

• actual: vector of observed class labels (0/1)
• predicted: vector of uncalibrated predictions
• model_idx: which calibration models should be implemented, 1=hist_scaled, 2=hist_transformed, 3=BBQ_scaled, 4=BBQ_transformed, 5=GUESS
• folds: number of folds in the cross-validation, Default: 10
• n_seeds: n_seeds determines how often random data set partition is repeated with varying seed
• nCores: nCores how many cores should be used during parallelisation. Default: 4

Returns

returns all trained calibration models that were built during the n_seeds-times repeated folds-CV.

Error values for each of the n_seeds CV runs are given.

Details

parallised execution over n_seeds