Data Analysis Part of 'IOHprofiler'
S3 sort function for DataSetList
S3 generic as.character operator for DataSet
Area Under Curve (Empirical Cumulative Dsitribution Function)
Bootstrapping for running time samples
S3 concatenation function for DataSet
S3 concatenation function for DataSetList
S3 generic cat operator for DataSet
Add unique identifiers to each DataSet in the provided DataSetList bas...
Verify that the credentials for DSCtool have been set
Check the format of data
Clean DataSetList object by concatenating DataSets
Constructor of S3 class 'DataSet'
S3 constructor of the 'DataSetList'
Empirical Cumulative Dsitribution Function of Runtime of a single data...
S3 generic == operator for DataSets
Function to get just the RT samples needed, without any formatting to ...
Generate dataframe of a single function/dimension pair
Generate dataframe containing the AUC for any ECDF-curves
Generate data for the cumulative difference plot.
Generate EAF-differences between each function and the remaining portf...
Generate differences between two EAFs
Generate dataframe consisting of the levelsets of the EAF
Generate dataframe consisting of the ECDF-equivalent based on the EAF
Generate dataframe of a the unaggregated values of individual algorith...
Generate dataframe of a single function/dimension pair
Nevergrad-dashboard based algorithm comparison
Generate dataframe of a single function/dimension pair
Generate dataframe of exactly 2 parameters, matched by running time
Generate dataframe of a single function/dimension pair
Generate dataframe of a single function/dimension pair for creating PD...
Generate dataframe of a single function/dimension pair
Get all algorithm ids present in a DataSetList
Get datatable of current color (and linestyle) scheme to file
Get colors according to the current colorScheme of the IOHanalyzer
Generate ECDF targets for a DataSetList
Get all dimensions present in a DataSetList
Perform omnibus statistical tests on the matrix of rankings from the D...
Perform post-hoc processing on data from DSCtool
Get the matrix of rankings using the DSCtool api for a DataSetList
Generation of default ECDF-targets
Get Expected RunTime
Get all function ids present in a DataSetList
Get all function names present in a DataSetList
Get all function values present in a DataSetList
Get Function Value condensed overview
Get Funtion Value Samples
Get Function Value Summary
Get function value matrix of the used dataset.
Get condensed overview of datasets
Get line styles according to the current styleScheme of the IOHanalyze...
Get the marginal contribution of an algorithm to a portfolio
Get the maximal running time
Get the list of available options for data from the OPTION ontology
Get the list of available options for data from the OPTION ontology
Get condensed overview of datasets
Get the parameter names of the algorithm
Get Parameter Value Samples
Get Parameter Value Summary
Get all parameter ids present in a DataSetList
Extract the position information from a datasetlist object
Get Runtime Value condensed overview
Get RunTime Sample
Get RunTime Summary
Get runtime matrix of the used dataset.
Get all runtime values present in a DataSetList
Get the shapley-values of a portfolio of algorithms
Get all options for a specific attribute which can be used to subset a...
Get all attributes which can be used to subset a DataSetList
Generate datatables of runtime or function value targets for a DataSet...
Glicko2 raning of algorithms
Template for creating plots in the IOHanalyzer-style
IOHanalyzer: Data Analysis Part of IOHprofiler
Deprecated function in package IOHanalyzer
Reduce the size of the data set by evenly subsampling the records
Get the ERT-values for all DataSets in a DataSetList at certain target...
Get the expected function-values for all DataSets in a DataSetList at ...
Performs a pairwise Kolmogorov-Smirnov test on the bootstrapped runnin...
Create EAF-based polygon plots
Create EAF-difference contour plots
General function for plotting within IOHanalyzer
Plot a heatmap according to the specifications from the Nevergrad dash...
Plot the cumulative difference plot given a DataSetList.
Plot expected function value-based comparison over multiple functions ...
Radarplot of the area under the aggregated ECDF-curve of a DataSetList...
Plot the empirical cumulative distriburtion as a function of the targe...
Plot the aggregated empirical cumulative distriburtion as a function o...
Plot histograms of the function values of a DataSetList at a certain t...
Plot FV-plots for multiple functions or dimensions
Plot the parameter values recorded in a DataSetList (aligned by budget...
Plot probablity density function of the function values of a DataSetLi...
Plot lineplot of the expected function values of a DataSetList
Create the PerformViz plot
Plot ERT-based comparison over multiple functions or dimensions
Radarplot of the area under the aggregated ECDF-curve of a DataSetList...
Plot the aggregated empirical cumulative distriburtion as a function o...
Plot the empirical cumulative distriburtion as a function of the runni...
Plot the aggregated empirical cumulative distriburtion as a function o...
Plot histograms of the runtimes of a DataSetList at a certain target f...
Plot ERT-plots for multiple functions or dimensions
Plot the parameter values recorded in a DataSetList (aligned by funcio...
Plot probablity mass function of the runtimes of a DataSetList at a ce...
Plot lineplot of the ERTs of a DataSetList
Create a candlestick plot of Glicko2-rankings
Plot a network graph showing the statistically different algorithms
Plot a heatmap showing the statistically different algorithms
S3 generic print operator for DataSet
S3 print function for DataSetList
Read .info files and extract information
Read Nevergrad data
Read Nevergrad data
Register an account to the DSCtool API
Create a shiny-server GUI to interactively use the IOHanalyzer
Save plotly figure in multiple format
Save DataTable in multiple formats
Scan *.info files for IOHProfiler or COCO
Function for generating sequences of function values
Function for generating sequences of runtime values
Set the colorScheme of the IOHanalyzer plots
Register an account to the DSCtool API
Estimator 'SP' for the Expected Running Time (ERT)
S3 extraction function for DataSetList
S3 subset function for DataSet
Filter a DataSetList by some criteria
S3 generic summary operator for DataSet
S3 summary function for DataSetList
The data analysis module for the Iterative Optimization Heuristics Profiler ('IOHprofiler'). This module provides statistical analysis methods for the benchmark data generated by optimization heuristics, which can be visualized through a web-based interface. The benchmark data is usually generated by the experimentation module, called 'IOHexperimenter'. 'IOHanalyzer' also supports the widely used 'COCO' (Comparing Continuous Optimisers) data format for benchmarking.
Useful links