GetNullAndPerfectScores function

Get Null And Perfect Scores

GetNullAndPerfectScores Calculates the score of Perfect and Null detectors scores. Perfect detector is one that outputs all true positives and no false positives. And Null detector is one that outputs no anomaly detections.

GetNullAndPerfectScores(data)

Arguments

• data: All dataset with training and test datasets and with at least timestamp, value and is.real.anomaly columns.

Returns

data.frame with null and perfect detectors scores for each profile.

Details

This function calculates the scores based on three different profiles. Each tp, fp, tn, fn label is associated with a weight to give a more realistic score. For the standard profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 1. For the reward_low_FP_rate profile weights are tp = 1, tn = 1, fp, = 0.22, and fn = 1. For the reward_low_FN_rate profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 2.

Examples

## Generate data
set.seed(100)
n <- 180
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE)
x[25] <- 200
x[150] <- 170
df <- data.frame(timestamp = 1:n, value = x)

# Add is.real.anomaly column
df$is.real.anomaly <- 0
df[c(25,80,150), "is.real.anomaly"] <- 1

# Get null and perfect scores
GetNullAndPerfectScores(df)

References

A. Lavin and S. Ahmad, “Evaluating Real-time Anomaly Detection Algorithms – the Numenta Anomaly Benchmark,” in 14th International Conference on Machine Learning and Applications (IEEE ICMLA’15), 2015.