# Anomaly detector custom expressions with time range

**URL:** <https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373>\
**Category:** Machine Learning\
**Tags:** anomaly-detection\
**Created:** [October 27, 2025, 2:20pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373 "2025-10-27T14:20:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![apaws06](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@apaws06](https://forum.opensearch.org/u/apaws06)\
**Post date:** [October 27, 2025, 2:20pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/1 "2025-10-27T14:20:01Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

v2.19

**Describe the issue** :

I would like to write a Custom expression in the “Anomaly Detection” to detect users login from outside working hours. The logic is very simple, to check certain type of username “\*adm” **and** the time range is outside UTC time from 10pm to 11a.m. The configuration below doesn’t seem working. Can someone check to see what I did wrong?

Any advice highly appreciated!

**Configuration** :

{  
“filtered\_output”: {  
“filter”: {  
“bool”: {  
“must”: [  
{  
“range”: {  
“@timestamp”: {  
“from”: “now-24h/d+22h”,  
“to”: “now-24h/d+13h”,  
“include\_lower”: false,  
“include\_upper”: false,  
“time\_zone”: “UTC”,  
“boost”: 1  
}  
}  
},  
{  
“wildcard”: {  
“username.keyword”: {  
“wildcard”: “\*adm”,  
“boost”: 1  
}  
}  
}  
],  
“adjust\_pure\_negative”: true,  
“boost”: 1  
}  
},  
“aggregations”: {  
“username”: {  
“value\_count”: {  
“field”: “username.keyword”  
}  
}  
}  
}  
}

**Relevant Logs or Screenshots** :

---

<div class="post-metadata">

**Author:** ![Anthony](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/anthony/32/9939_2.png) [@Anthony](https://forum.opensearch.org/u/Anthony)\
**Post date:** [October 28, 2025, 11:49am UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/2 "2025-10-28T11:49:27Z")

</div>

@apaws06 You can use a script to achieve this.

I would recommend to run this in devtools first and ensure that you are getting `count_logins.value` \>0 in some of the buckets (assuming you have these anomalies already in the data set, otherwise test this on a sample set first).

```auto
POST logins-demo/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        { "wildcard": { "username": "*adm" } },
        {
          "script": {
            "script": {
              "lang": "painless",
              "source": "def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11"
            }
          }
        },
        { "range": { "@timestamp": { "gte": "now-2d", "lt": "now" } } }
      ]
    }
  },
  "aggs": {
    "buckets": {
      "date_histogram": { "field": "@timestamp", "fixed_interval": "5m" },
      "aggs": {
        "count_logins": { "value_count": { "field": "username" } } // <-- use "username" here
      }
    }
  }
}

```

You should see something like this:

```auto
"hits": {
    "total": {
      "value": 6,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "buckets": {
      "buckets": [
        {
          "key_as_string": "2025-10-27T22:15:00.000Z",
          "key": 1761603300000,
          "doc_count": 2,
          "count_logins": {
            "value": 2
          }
        },
        {
          "key_as_string": "2025-10-27T22:20:00.000Z",
          "key": 1761603600000,
          "doc_count": 0,
          "count_logins": {
            "value": 0
          }
        },

```

If you are not getting any hits, this could be a result of a mapping issue.

In my testing the final detector in `.opendistro-anomaly-detectors` index (accessible using admin cert and key) looks like this:

```auto
{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : ".opendistro-anomaly-detectors",
        "_id" : "VdSNKpoB7iQw7zyGj4oT",
        "_score" : 1.0,
        "_source" : {
          "name" : "detector1",
          "description" : "",
          "time_field" : "@timestamp",
          "indices" : [
            "logins-demo"
          ],
          "filter_query" : {
            "bool" : {
              "filter" : [
                {
                  "bool" : {
                    "filter" : [
                      {
                        "wildcard" : {
                          "username" : {
                            "wildcard" : "*adm",
                            "boost" : 1.0
                          }
                        }
                      },
                      {
                        "script" : {
                          "script" : {
                            "source" : "def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11",
                            "lang" : "painless"
                          },
                          "boost" : 1.0
                        }
                      }
                    ],
                    "adjust_pure_negative" : true,
                    "boost" : 1.0
                  }
                }
              ],
              "adjust_pure_negative" : true,
              "boost" : 1.0
            }
          },
          "window_delay" : {
            "period" : {
              "interval" : 1,
              "unit" : "Minutes"
            }
          },
          "shingle_size" : 1,
          "schema_version" : 0,
          "feature_attributes" : [
            {
              "feature_id" : "UtSNKpoB7iQw7zyGjorR",
              "feature_name" : "feature1",
              "feature_enabled" : true,
              "aggregation_query" : {
                "count_logins" : {
                  "value_count" : {
                    "field" : "username"
                  }
                }
              }
            }
          ],
          "recency_emphasis" : 2560,
          "history" : 40,
          "ui_metadata" : {
            "features" : {
              "feature1" : {
                "featureType" : "custom_aggs"
              }
            },
            "filters" : [
              {
                "query" : "{\n \"bool\": {\n \"filter\": [\n { \"wildcard\": { \"username\": \"*adm\" } },\n {\n \"script\": {\n \"script\": {\n \"source\": \"def h = doc['@timestamp'].value.getHour(); h >= 22 || h < 11\",\n \"lang\": \"painless\"\n }\n }\n }\n ]\n }\n}",
                "label" : "",
                "filterType" : "custom_filter",
                "fieldInfo" : [],
                "fieldValue" : "",
                "operator" : "is"
              }
            ]
          },
          "last_update_time" : 1761651057589,
          "user" : {
            "name" : "admin",
            "backend_roles" : [
              "admin"
            ],
            "roles" : [
              "security_rest_api_access",
              "all_access"
            ],
            "custom_attribute_names" : [],
            "user_requested_tenant" : " __user__",
            "user_requested_tenant_access" : "NONE"
          },
          "detection_interval" : {
            "period" : {
              "interval" : 10,
              "unit" : "Minutes"
            }
          },
          "detector_type" : "SINGLE_ENTITY",
          "rules" : [
            {
              "action" : "IGNORE_ANOMALY",
              "conditions" : [
                {
                  "feature_name" : "feature1",
                  "threshold_type" : "ACTUAL_OVER_EXPECTED_RATIO",
                  "operator" : "LTE",
                  "value" : 0.2
                }
              ]
            },
            {
              "action" : "IGNORE_ANOMALY",
              "conditions" : [
                {
                  "feature_name" : "feature1",
                  "threshold_type" : "EXPECTED_OVER_ACTUAL_RATIO",
                  "operator" : "LTE",
                  "value" : 0.2
                }
              ]
            }
          ]
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![apaws06](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@apaws06](https://forum.opensearch.org/u/apaws06)\
**Post date:** [October 29, 2025, 3:32am UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/3 "2025-10-29T03:32:44Z")

</div>

Hello Anthony,

Thanks for the quick feedback. I got hits results back from the Dev Tools; however, when I created a Anomaly Detector with one Feature with Custom Expression base on “count\_logins” for “username” field, I got the error below:  
**Issues found in the model configuration: The “count\_logins” Feature has an invalid query causing a runtime exception**

Please let me know what I did wrong for the **“count\_logins”** Feature below:

{  
“filtered\_output”: {  
“filter”: {  
“bool”: {  
“must”: [  
{  
“script” : {  
“script” : {  
“source” : “def h = doc[‘@timestamp’].value.getHour(); h \>= 23 || h \< 12”,  
“lang” : “painless”  
},  
“boost” : 1.0  
}  
},  
{  
“wildcard”: {  
“username.keyword”: {  
“wildcard”: “\*adm”,  
“boost”: 1  
}  
}  
}  
],  
“adjust\_pure\_negative”: true,  
“boost”: 1  
}  
},  
“aggs”: {  
“buckets”: {  
“date\_histogram”: { “field”: “@timestamp”, “fixed\_interval”: “10m” },  
“aggs”: {  
“count\_logins”: { “value\_count”: { “field”: “username.keyword” } }  
}  
}  
}  
}  
}

Thank you in advance,

Andre

---

<div class="post-metadata">

**Author:** ![Anthony](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/anthony/32/9939_2.png) [@Anthony](https://forum.opensearch.org/u/Anthony)\
**Post date:** [October 29, 2025, 10:32am UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/4 "2025-10-29T10:32:06Z")

</div>

@apaws06 Can you try to create a detector using API:

```auto
POST _plugins/_anomaly_detection/detectors
{
  "name": "off-hours-adm-logins",
  "description": "Counts *adm usernames during 22:00–11:00 UTC",
  "time_field": "@timestamp",
  "indices": ["logins-demo"],
  "feature_attributes": [
    {
      "feature_name": "count_logins",
      "feature_enabled": true,
      "aggregation_query": {
        "count_logins": {
          "value_count": { "field": "username" }
        }
      }
    }
  ],
  "filter_query": {
    "bool": {
      "filter": [
        { "wildcard": { "username": "*adm" } },
        {
          "bool": {
            "should": [
              { "range": { "@timestamp": { "gte": "now-1d/d+22h", "lt": "now/d", "time_zone": "UTC" } } },
              { "range": { "@timestamp": { "gte": "now/d", "lt": "now/d+11h", "time_zone": "UTC" } } }
            ],
            "minimum_should_match": 1
          }
        }
      ]
    }
  },
  "detection_interval": { "period": { "interval": 5, "unit": "Minutes" } },
  "window_delay": { "period": { "interval": 1, "unit": "Minutes" } }
}

```

This will provide the ID of the detector, using this ID you can then start it with:

```auto
POST _plugins/_anomaly_detection/detectors/PUT_DETECTOR_ID_HERE/_start

```

NOTE: In my case field: `username` is already mapped as `keyword`, can you provide your mappings of the index if this doesn’t work for you.

---

<div class="post-metadata">

**Author:** ![apaws06](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@apaws06](https://forum.opensearch.org/u/apaws06)\
**Post date:** [October 31, 2025, 5:11pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/5 "2025-10-31T17:11:48Z")

</div>

Anthony,

This is awesome. I got the Anomaly Detector working as suggested and it detects the anomalies within the time range specified. So basically, the filter query for time range is moved to Detector settings Data filter with Custom expression and the Features just do normal value\_count; however, the new issue right now is the detector is in “Initializing” state after I added the ipAddress associating with the username in the “Additional settings” Categorical fields as below somehow broke the Detector API :

“category\_field”: [  
“ipAddress”,  
“username.keyword”  
],

Can you try your end to see similar issue once added the Categorical fields?

Thanks,

Andre

---

<div class="post-metadata">

**Author:** ![Anthony](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/anthony/32/9939_2.png) [@Anthony](https://forum.opensearch.org/u/Anthony)\
**Post date:** [November 3, 2025, 12:37pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/6 "2025-11-03T12:37:07Z")

</div>

@apaws06 Most likely the issue is with lack of data points in the buckets for the detector to initialise. As the detector is building a separate models with a per user/ip baseline. You can run a search similar to the below to see what the detector would be able to see:

```auto
POST logins/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        { "term": { "username": "opsadm" } },
        { "term": { "ipAddress": "10.10.2.20" } },
        { "range": { "@timestamp": { "gte": "now-6h", "lt": "now" } } }
      ]
    }
  },
  "aggs": {
    "per_10m": {
      "date_histogram": { "field": "@timestamp", "fixed_interval": "10m" },
      "aggs": { "cnt": { "value_count": { "field": "username" } } }
    }
  }
}

```

Using categorical fields can be tricky to manage for a number of reasons.

Cardinality multiplies: entities = `unique(username) × unique(ipAddress)`. More entities means more memory and more data needed. Each entity needs consecutive buckets to initialize.  
Data sparsity: if most (user,IP) pairs are rare, many models won’t have enough points and the detector can sit in INIT. It would probably be better to use one categorical field in that case.

Hope this helps

---

<div class="post-metadata">

**Author:** ![apaws06](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@apaws06](https://forum.opensearch.org/u/apaws06)\
**Post date:** [November 5, 2025, 11:34pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/7 "2025-11-05T23:34:45Z")

</div>

Anthony,

It works now that I just added one categorical field (username.keyword). It took couple hours to turn from “Initializing” state to “Running” state. The Anomaly Detector now have username when anomalies occurred within the specified time range.

Thank you and I really appreciate your help.

Andre

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [January 4, 2026, 11:35pm UTC](https://forum.opensearch.org/t/anomaly-detector-custom-expressions-with-time-range/27373/8 "2026-01-04T23:35:04Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.