Abstract Class for Confidence Intervals
Base class for confidence interval measures. See section Inheriting on how to add a new method.
The aggregator of the wrapped measure is ignored, as the inheriting CI dictates how the point estimate is constructed. If a measure for which to calculate a CI has $obs_loss but also a $trafo, (such as RMSE), the delta method is used to obtain confidence intervals.
alpha :: numeric(1)
The desired alpha level. This is initialized to .
within_range :: logical(1)
Whether to restrict the confidence interval within the range of possible values. This is initialized to TRUE.
To define a new CI method, inherit from the abstract base class and implement the private method: ci: function(tbl: data.table, rr: ResampleResult, param_vals: namedlist()) -> numeric(3)
If requires_obs_loss is set to TRUE, tbl contains the columns loss, row_id and iteration, which are the pointwise loss, Otherwise, tbl contains the result of rr$score() with the name of the loss column set to "loss". the identifier of the observation and the resampling iteration. It should return a vector containing the estimate, lower and upper boundary in that order.
In case the confidence interval is not of the form (estimate, estimate - z * se, estimate + z * se)
it is also necessary to implement the private method: .trafo: function(ci: numeric(3), measure: Measure) -> numeric(3)
Which receives a confidence interval for a pointwise loss (e.g. squared-error) and transforms it according to the transformation measure$trafo (e.g. sqrt to go from mse to rmse).
mlr3::Measure -> MeasureAbstractCi
resamplings: (character())
On which resampling classes this method can operate.
measure: (Measure)
new()Creates a new instance of this R6 class.
MeasureAbstractCi$new(
measure = NULL,
param_set = ps(),
packages = character(),
resamplings,
label,
delta_method = FALSE,
requires_obs_loss = TRUE
)
measure: (Measure)
The measure for which to calculate a confidence interval. Must have `$obs_loss`.
param_set: (ParamSet)
Set of hyperparameters.
packages: (character())
Set of required packages. A warning is signaled by the constructor if at least one of the packages is not installed, but loaded (not attached) later on-demand via `requireNamespace()`.
resamplings: (character())
To which resampling classes this measure can be applied.
label: (character(1))
Label for the new instance.
delta_method: (logical(1))
Whether to use the delta method for measures (such RMSE) that have a trafo.
requires_obs_loss: (logical(1))
Whether the inference method requires a pointwise loss function.
aggregate()Obtain a point estimate, as well as lower and upper CI boundary.
MeasureAbstractCi$aggregate(rr)
rr: (ResampleResult)
The resample result.
named numeric(3)
clone()The objects of this class are cloneable with this method.
MeasureAbstractCi$clone(deep = FALSE)
deep: Whether to make a deep clone.
Useful links