Anomaly detector custom expressions with time range

Plugins Machine Learning
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

v2.19

Describe the issue:

I would like to write a Custom expression in the “Anomaly Detection” to detect users login from outside working hours. The logic is very simple, to check certain type of username “*adm” and the time range is outside UTC time from 10pm to 11a.m. The configuration below doesn’t seem working. Can someone check to see what I did wrong?

Any advice highly appreciated!

Configuration:

{
“filtered_output”: {
“filter”: {
“bool”: {
“must”: [
{
“range”: {
“@timestamp”: {
“from”: “now-24h/d+22h”,
“to”: “now-24h/d+13h”,
“include_lower”: false,
“include_upper”: false,
“time_zone”: “UTC”,
“boost”: 1
}
}
},
{
“wildcard”: {
“username.keyword”: {
“wildcard”: “*adm”,
“boost”: 1
}
}
}
],
“adjust_pure_negative”: true,
“boost”: 1
}
},
“aggregations”: {
“username”: {
“value_count”: {
“field”: “username.keyword”
}
}
}
}
}

Relevant Logs or Screenshots:

2

@apaws06 You can use a script to achieve this.

I would recommend to run this in devtools first and ensure that you are getting count_logins.value >0 in some of the buckets (assuming you have these anomalies already in the data set, otherwise test this on a sample set first).

POST logins-demo/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        { "wildcard": { "username": "*adm" } },
        {
          "script": {
            "script": {
              "lang": "painless",
              "source": "def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11"
            }
          }
        },
        { "range": { "@timestamp": { "gte": "now-2d", "lt": "now" } } }
      ]
    }
  },
  "aggs": {
    "buckets": {
      "date_histogram": { "field": "@timestamp", "fixed_interval": "5m" },
      "aggs": {
        "count_logins": { "value_count": { "field": "username" } }   // <-- use "username" here
      }
    }
  }
}

You should see something like this:

"hits": {
    "total": {
      "value": 6,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "buckets": {
      "buckets": [
        {
          "key_as_string": "2025-10-27T22:15:00.000Z",
          "key": 1761603300000,
          "doc_count": 2,
          "count_logins": {
            "value": 2
          }
        },
        {
          "key_as_string": "2025-10-27T22:20:00.000Z",
          "key": 1761603600000,
          "doc_count": 0,
          "count_logins": {
            "value": 0
          }
        },

If you are not getting any hits, this could be a result of a mapping issue.

In my testing the final detector in .opendistro-anomaly-detectors index (accessible using admin cert and key) looks like this:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : ".opendistro-anomaly-detectors",
        "_id" : "VdSNKpoB7iQw7zyGj4oT",
        "_score" : 1.0,
        "_source" : {
          "name" : "detector1",
          "description" : "",
          "time_field" : "@timestamp",
          "indices" : [
            "logins-demo"
          ],
          "filter_query" : {
            "bool" : {
              "filter" : [
                {
                  "bool" : {
                    "filter" : [
                      {
                        "wildcard" : {
                          "username" : {
                            "wildcard" : "*adm",
                            "boost" : 1.0
                          }
                        }
                      },
                      {
                        "script" : {
                          "script" : {
                            "source" : "def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11",
                            "lang" : "painless"
                          },
                          "boost" : 1.0
                        }
                      }
                    ],
                    "adjust_pure_negative" : true,
                    "boost" : 1.0
                  }
                }
              ],
              "adjust_pure_negative" : true,
              "boost" : 1.0
            }
          },
          "window_delay" : {
            "period" : {
              "interval" : 1,
              "unit" : "Minutes"
            }
          },
          "shingle_size" : 1,
          "schema_version" : 0,
          "feature_attributes" : [
            {
              "feature_id" : "UtSNKpoB7iQw7zyGjorR",
              "feature_name" : "feature1",
              "feature_enabled" : true,
              "aggregation_query" : {
                "count_logins" : {
                  "value_count" : {
                    "field" : "username"
                  }
                }
              }
            }
          ],
          "recency_emphasis" : 2560,
          "history" : 40,
          "ui_metadata" : {
            "features" : {
              "feature1" : {
                "featureType" : "custom_aggs"
              }
            },
            "filters" : [
              {
                "query" : "{\n  \"bool\": {\n    \"filter\": [\n      { \"wildcard\": { \"username\": \"*adm\" } },\n      {\n        \"script\": {\n          \"script\": {\n            \"source\": \"def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11\",\n            \"lang\": \"painless\"\n          }\n        }\n      }\n    ]\n  }\n}",
                "label" : "",
                "filterType" : "custom_filter",
                "fieldInfo" : [ ],
                "fieldValue" : "",
                "operator" : "is"
              }
            ]
          },
          "last_update_time" : 1761651057589,
          "user" : {
            "name" : "admin",
            "backend_roles" : [
              "admin"
            ],
            "roles" : [
              "security_rest_api_access",
              "all_access"
            ],
            "custom_attribute_names" : [ ],
            "user_requested_tenant" : "__user__",
            "user_requested_tenant_access" : "NONE"
          },
          "detection_interval" : {
            "period" : {
              "interval" : 10,
              "unit" : "Minutes"
            }
          },
          "detector_type" : "SINGLE_ENTITY",
          "rules" : [
            {
              "action" : "IGNORE_ANOMALY",
              "conditions" : [
                {
                  "feature_name" : "feature1",
                  "threshold_type" : "ACTUAL_OVER_EXPECTED_RATIO",
                  "operator" : "LTE",
                  "value" : 0.2
                }
              ]
            },
            {
              "action" : "IGNORE_ANOMALY",
              "conditions" : [
                {
                  "feature_name" : "feature1",
                  "threshold_type" : "EXPECTED_OVER_ACTUAL_RATIO",
                  "operator" : "LTE",
                  "value" : 0.2
                }
              ]
            }
          ]
        }
      }
    ]
  }
}
3

Hello Anthony,

Thanks for the quick feedback. I got hits results back from the Dev Tools; however, when I created a Anomaly Detector with one Feature with Custom Expression base on “count_logins” for “username” field, I got the error below:
Issues found in the model configuration: The “count_logins” Feature has an invalid query causing a runtime exception

Please let me know what I did wrong for the “count_logins” Feature below:

{
“filtered_output”: {
“filter”: {
“bool”: {
“must”: [
{
“script” : {
“script” : {
“source” : “def h = doc[‘@timestamp’].value.getHour(); h >= 23 || h < 12”,
“lang” : “painless”
},
“boost” : 1.0
}
},
{
“wildcard”: {
“username.keyword”: {
“wildcard”: “*adm”,
“boost”: 1
}
}
}
],
“adjust_pure_negative”: true,
“boost”: 1
}
},
“aggs”: {
“buckets”: {
“date_histogram”: { “field”: “@timestamp”, “fixed_interval”: “10m” },
“aggs”: {
“count_logins”: { “value_count”: { “field”: “username.keyword” } }
}
}
}
}
}

Thank you in advance,

Andre

4

@apaws06 Can you try to create a detector using API:

POST _plugins/_anomaly_detection/detectors
{
  "name": "off-hours-adm-logins",
  "description": "Counts *adm usernames during 22:00–11:00 UTC",
  "time_field": "@timestamp",
  "indices": ["logins-demo"],
  "feature_attributes": [
    {
      "feature_name": "count_logins",
      "feature_enabled": true,
      "aggregation_query": {
        "count_logins": {
          "value_count": { "field": "username" }
        }
      }
    }
  ],
  "filter_query": {
    "bool": {
      "filter": [
        { "wildcard": { "username": "*adm" } },
        {
          "bool": {
            "should": [
              { "range": { "@timestamp": { "gte": "now-1d/d+22h", "lt": "now/d",      "time_zone": "UTC" } } },
              { "range": { "@timestamp": { "gte": "now/d",       "lt": "now/d+11h", "time_zone": "UTC" } } }
            ],
            "minimum_should_match": 1
          }
        }
      ]
    }
  },
  "detection_interval": { "period": { "interval": 5, "unit": "Minutes" } },
  "window_delay":       { "period": { "interval": 1, "unit": "Minutes" } }
}

This will provide the ID of the detector, using this ID you can then start it with:

POST _plugins/_anomaly_detection/detectors/PUT_DETECTOR_ID_HERE/_start

NOTE: In my case field: username is already mapped as keyword, can you provide your mappings of the index if this doesn’t work for you.

5

Anthony,

This is awesome. I got the Anomaly Detector working as suggested and it detects the anomalies within the time range specified. So basically, the filter query for time range is moved to Detector settings Data filter with Custom expression and the Features just do normal value_count; however, the new issue right now is the detector is in “Initializing” state after I added the ipAddress associating with the username in the “Additional settings” Categorical fields as below somehow broke the Detector API :

“category_field”: [
“ipAddress”,
“username.keyword”
],

Can you try your end to see similar issue once added the Categorical fields?

Thanks,

Andre

6

@apaws06 Most likely the issue is with lack of data points in the buckets for the detector to initialise. As the detector is building a separate models with a per user/ip baseline. You can run a search similar to the below to see what the detector would be able to see:

POST logins/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        { "term": { "username": "opsadm" } },
        { "term": { "ipAddress": "10.10.2.20" } },
        { "range": { "@timestamp": { "gte": "now-6h", "lt": "now" } } }
      ]
    }
  },
  "aggs": {
    "per_10m": {
      "date_histogram": { "field": "@timestamp", "fixed_interval": "10m" },
      "aggs": { "cnt": { "value_count": { "field": "username" } } }
    }
  }
}

Using categorical fields can be tricky to manage for a number of reasons.

Cardinality multiplies: entities = unique(username) × unique(ipAddress). More entities means more memory and more data needed. Each entity needs consecutive buckets to initialize.
Data sparsity: if most (user,IP) pairs are rare, many models won’t have enough points and the detector can sit in INIT. It would probably be better to use one categorical field in that case.

Hope this helps

7

Anthony,

It works now that I just added one categorical field (username.keyword). It took couple hours to turn from “Initializing” state to “Running” state. The Anomaly Detector now have username when anomalies occurred within the specified time range.

Thank you and I really appreciate your help.

Andre

1 Like