MeasureClassif function

Classification Measure

This measure specializes Measure for classification problems:

• task_type is set to "classif".
• Possible values for predict_type are "response" and "prob".

Predefined measures can be found in the dictionary mlr_measures . The default measure for classification is classif.ce.

See Also

• Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter2/data_and_basic_modeling.html#sec-eval

• Package list("mlr3measures") for the scoring functions. Dictionary of Measures : mlr_measures

  as.data.table(mlr_measures) for a table of available Measures in the running session (depending on the loaded packages).

• Extension packages for additional task types:

  • list("mlr3proba") for probabilistic supervised regression and survival analysis.
  • list("mlr3cluster") for unsupervised clustering.

Other Measure: Measure, MeasureRegr, MeasureSimilarity, mlr_measures, mlr_measures_aic, mlr_measures_bic, mlr_measures_classif.costs, mlr_measures_debug_classif, mlr_measures_elapsed_time, mlr_measures_internal_valid_score, mlr_measures_oob_error, mlr_measures_regr.rsq, mlr_measures_selected_features

Super class

mlr3::Measure -> MeasureClassif

Methods

Public methods

• list("MeasureClassif$new()")

• list("MeasureClassif$clone()")

Method new()

Creates a new instance of this R6 class.

Usage

MeasureClassif$new(
  id,
  param_set = ps(),
  range,
  minimize = NA,
  average = "macro",
  aggregator = NULL,
  properties = character(),
  predict_type = "response",
  predict_sets = "test",
  task_properties = character(),
  packages = character(),
  label = NA_character_,
  man = NA_character_
)

Arguments

• id: (character(1))

   Identifier for the new instance.
  

• param_set: (paradox::ParamSet )

   Set of hyperparameters.
  

• range: (numeric(2))

   Feasible range for this measure as `c(lower_bound, upper_bound)`. Both bounds may be infinite.
  

• minimize: (logical(1))

   Set to `TRUE` if good predictions correspond to small values, and to `FALSE` if good predictions correspond to large values. If set to `NA` (default), tuning this measure is not possible.
  

• average: (character(1))

   How to average multiple Prediction s from a ResampleResult .
   
   The default, `"macro"`, calculates the individual performances scores for each Prediction and then uses the function defined in `$aggregator` to average them to a single number.
   
   If set to `"micro"`, the individual Prediction objects are first combined into a single new Prediction object which is then used to assess the performance. The function in `$aggregator` is not used in this case.
  

• aggregator: (function())

   Function to aggregate over multiple iterations. The role of this function depends on the value of field `"average"`:
   
    * `"macro"`: A numeric vector of scores (one per iteration) is passed. The aggregate function defaults to `mean()` in this case.
    * `"micro"`: The `aggregator` function is not used. Instead, predictions from multiple iterations are first combined and then scored in one go.
    * `"custom"`: A ResampleResult is passed to the aggregate function.
  

• properties: (character())

   Properties of the measure. Must be a subset of mlr_reflections$measure_properties . Supported by `mlr3`:
   
    * `"requires_task"` (requires the complete Task ),
    * `"requires_learner"` (requires the trained Learner ),
    * `"requires_model"` (requires the trained Learner , including the fitted model),
    * `"requires_train_set"` (requires the training indices from the Resampling ), and
    * `"na_score"` (the measure is expected to occasionally return `NA` or `NaN`).
    * `"primary_iters"` (the measure explictly handles resamplings that only use a subset of their iterations for the point estimate).
    * `"requires_no_prediction"` (No prediction is required; This usually means that the measure extracts some information from the learner state.).
  

• predict_type: (character(1))

   Required predict type of the Learner . Possible values are stored in mlr_reflections$learner_predict_types .
  

• predict_sets: (character())

   Prediction sets to operate on, used in `aggregate()` to extract the matching `predict_sets` from the ResampleResult . Multiple predict sets are calculated by the respective Learner during `resample()`/`benchmark()`. Must be a non-empty subset of `{"train", "test", "internal_valid"}`. If multiple sets are provided, these are first combined to a single prediction object. Default is `"test"`.
  

• task_properties: (character())

   Required task properties, see Task .
  

• packages: (character())

   Set of required packages. A warning is signaled by the constructor if at least one of the packages is not installed, but loaded (not attached) later on-demand via `requireNamespace()`.
  

• label: (character(1))

   Label for the new instance.
  

• man: (character(1))

   String in the format `[pkg]::[topic]` pointing to a manual page for this object. The referenced help package can be opened via method `$help()`.
  

Method clone()

The objects of this class are cloneable with this method.

Usage

MeasureClassif$clone(deep = FALSE)

Arguments

• deep: Whether to make a deep clone.