Are there pre-made Anomaly Detection detectors?

Plugins Machine Learning
1

I want to implement anomaly detection, but I want to know if there are any default detectors or pre-made detectors?

2

The challenge with pre-built detectors is that they will be specific to the data, or more specifically to the schema, that is being analyzed. For common and consistent datasources this can make sense. For example… prebuilt detectors for Metricbeat’s System module.

We are working on some pre-built anomaly detections for ElastiFlow, similar to what we provide for Elastic’s ML features. However these will be specific to the 'CODEX" schema we use in ElastiFlow when sending data to OpenSearch. These should be available by the end of July.

1 Like
3

Thank you!

My issue is building effective anomaly detectors. Besides Anomaly Detection - Open Distro Documentation , is there a resource in building effective anomaly detectors. I do not know if having multiple features I include are good or not.

4

Hi kanrab,
I am one of the contributors to the anomaly detection plugin. Here’s some information that might be useful.

Since the data for anomaly detection is different, and even on the same data, the intents/definitions of anomalies can be different, there is no general way to make one detector that works on all data. The users must define what a detector does for their specific problems.

The very beginning is to define what is being monitored, error counts? p99 latency? a combination? For an example, there is a blog Preprocess logs for anomaly detection in Amazon OpenSearch | AWS Big Data Blog . As for which ones are good, the suggestion is just to create different detectors with different sets of features, let them run on the data, and see which detectors better solve the problems.