Using Anomaly Detection to detect sudden increases or decreases in the amount of logs received for an index

Plugins Machine Learning
1

Hi,

We have a need to detect sudden changes to the amount of logs received for certain indices. One idea we had was using the OpenSearch Anomaly Detection plugin to perhaps do a count() on the @timestamp field.

We tested this and got some results back from the historical analysis, but the detections were a little to picky for us. The anomalies were created when we got an increase around about 2-5%, instead we would like to get an alert when it differs around 10% or more.

Is this possible? Thanks

1 Like
2

Humm. How much does your data typically vary on the count()?

Also, what is your interval - is that set too tight?

3

trying different intervals might help. Also, you can use alerting since you know the threshold to trigger.