Anomaly detection: Why is this not an anomaly?


I couldn’t find a category for AD or plugins in general, so posting here instead.
Small disclaimer: This is using ES OSS with latest OpenDistro plugins.

I started my testing detector almost a month ago, so I have some decent data.
It’s easier if I just show you:

I’m confused as to why the fluctuations on January 12, 22-26 and 31 are not considered anomalous.

The detector monitors syslog messages from network equipment and the feature looks at BFD Link Down events. It has a 10 minute interval and 1 minute window delay. Window size is 2.

1 Like

So this is less of a technical answer and more of a arbitrary experience point. Anomaly detection is an imprecise science that takes a bit of tuning. What may be happening here is: there may be enough anomalies that it has started to detect them as normal now. I am not an expert in this space but that is what I experienced with another Random Cut Forest (RCF) model I tried to deploy.

Curious to hear other peoples thoughts. Also, one thing that you may want to try is messing with the confidence intervals and bucketing.