Hello there,
We’ve recently upgraded our ES cluster and now have the ability to use anomoly detection. I’ve read the Open Distro documentation but I’m not sure how to get the anomoly detection working for my use case.
Is it possible to configure a detector so that when it runs it checks the same time for the last two or three weeks? Is that part of the Windows size when setting up the model configuration?
I want to set up a detector which checks the number of invocations to see if we have an service interruption or not (# of invocations). Can I set up an anomoly detector so when it scans it looks 10 minutes in the past and also the same window for the past 2 to 4 weeks?
@jberto78 Thanks for your interest in anomaly detection (AD) plugin. Currently AD plugin supports detecting time series data in realtime way. The model will learn from historical data and it doesn’t support checking the same window for past weeks. But we are happy to help build some new feature to solve your problem. Can you elaborate your use case? Seems you want to analyze trend/weekly pattern?
Hi @ylwu; apologies for the delayed response to your inquiry. My use case is as follows:
I want to be able to use the Anomaly detection to check for invocations of a Lambda. I would like it to aggregate in buckets so that it looks at the same period in time going back three weeks. Ultimately I want the anomaly detection to see if the ‘count’ of invocations for today is within the threshold of the count for the same time last week and the week before that. If it detects a anomaly (today’s count is 10% lower than the last two weeks) then to alert on it.
I’m able to get something similar to what I want in the ‘Dev Tools’ console (see below) as far as generating a buckets:
“aggregations” : {
“CallVolume” : {
“buckets” : [
{
“key” : “2021-09-16T20:20:38.774Z-2021-09-16T20:25:38.774Z”,
“from” : 1.631823638774E12,
“from_as_string” : “2021-09-16T20:20:38.774Z”,
“to” : 1.631823938774E12,
“to_as_string” : “2021-09-16T20:25:38.774Z”,
“doc_count” : 177
},
{
“key” : “2021-09-23T20:20:38.774Z-2021-09-23T20:25:38.774Z”,
“from” : 1.632428438774E12,
“from_as_string” : “2021-09-23T20:20:38.774Z”,
“to” : 1.632428738774E12,
“to_as_string” : “2021-09-23T20:25:38.774Z”,
“doc_count” : 157
},
{
“key” : “2021-09-30T20:20:38.774Z-*”,
“from” : 1.633033238774E12,
“from_as_string” : “2021-09-30T20:20:38.774Z”,
“doc_count” : 143
}
]
}
}