Differences between real-time and historical anomaly detection

Plugins Machine Learning
1

I’ve been playing around with the historical data anomaly detection released in 1.13.0. Really cool postmortem-style analysis of network traffic or whatever your application is.

I’ve noticed a few configuration options that exist in configuring real-time detectors and are missing from historical detectors:

  • Data filter (under detector data source configuration)
  • Category field for high cardinality (search for “cardinality” here)

Both of these (but particularly the category field are, in my opinion, really essential peices of the anomaly detector plugin. Does anybody here have any input on this as to why they are unavailable for historical data analysis? Is this something I should submit as an enhancement request on opendistro-for-elasticsearch/anomaly-detection or opendistro-for-elasticsearch/anomaly-detection-kibana-plugin or both?

Thanks,

SG

2

hi, @tlacuache thanks for your interest in historical detector feature. In this ODFE 1.13 release, we don’t include high cardianlity(category field) support. We are building unified flow to support both realtime and historical detection under the same detector to make the workflow smooth. This is the RFC RFC: Unified flow for realtime and historical anomaly detection · Issue #380 · opendistro-for-elasticsearch/anomaly-detection · GitHub, welcome any comments/suggestions.

1 Like