Using Anomaly Detection to detect sudden increases or decreases in the amount of logs received for an index


We have a need to detect sudden changes to the amount of logs received for certain indices. One idea we had was using the OpenSearch Anomaly Detection plugin to perhaps do a count() on the @timestamp field.

We tested this and got some results back from the historical analysis, but the detections were a little to picky for us. The anomalies were created when we got an increase around about 2-5%, instead we would like to get an alert when it differs around 10% or more.

Is this possible? Thanks

Humm. How much does your data typically vary on the count()?

Also, what is your interval - is that set too tight?

trying different intervals might help. Also, you can use alerting since you know the threshold to trigger.