OpenSearch Anomaly Detection by error rate

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

we use OpenSearch Dashboard v 2.15.0

Describe the issue:
We want to implement “anomaly detection” by error rate.
Question: which field (or expression) do we need to specify in feature configuration (within Model configuration section) for to implement error count by rate (instead of currently used level.keyword field, i.e. LOG level)?

Configuration:
For to filter only log messages with non-empty exception, we’ve specified the following filter expression:

{
  "exists": {
    "field": "exception.exception_class"
  }
}

Within Feature Model configuration we use value_count method with level.keyword field

Here’s our log message structure (OpenSearch message structure):

{
  "_index": "ourIndexName",
  "_id": "alphaNumericId",
  "_version": 1,
  "_score": null,
  "_source": {
    "Attributes": {
      "Tenant": "",
      "client.device.session_id": "sessionIdHere",
      "error.id": "errorId",
      "forwardedUniqueId": "some uniqueId",
      "span_id": "aSpanId",
      "trace.id": "anId",
      "trace_flags": "00",
      "trace_id": "anId",
      "transaction.id": "anId",
      "uniqueid": "anId"
    },
    "exception": {
      "exception_class": "IllegalStateException",
      "exception_message": "some exception message",
      "stacktrace": "long stack trace"
    },
    "line_number": 68,
    "class": "a ClassName",
    "@version": 1,
    "source_host": "someHost",
    "message": "aMessage",
    "thread_name": "someThreadName",
    "@timestamp": "timestamp",
    "level": "ERROR",
    "file": "SomeFileName.java",
    "method": "someMethodName",
    "logger_name": "someFullyQualifiedLoggerName",
    "@log_name": "theLogName"
  },
  "fields": {
    "@timestamp": [
      "timestamp"
    ]
  },
  "highlight": {
    "exception.stacktrace": [
      "the Exception stack trace"
    ]
  },
  "sort": [
    1234567890123
  ]
}

And here’s the screenshot of our current feature configuration:

image
image588×705 38.2 KB