Trigger by one of buckets value

Plugins Alerting
,
1

Hi. I really don’t understand from documentation how to use “Per bucket monitor” monitor type and buckets_path to build a condition for a trigger with value from one particular bucket.
I have following response in a query section:

{
    "_shards": {
        "total": 413,
        "failed": 0,
        "successful": 413,
        "skipped": 147
    },
    "hits": {
        "hits": [],
        "total": {
            "value": 20,
            "relation": "eq"
        },
        "max_score": null
    },
    "took": 155,
    "timed_out": false,
    "aggregations": {
        "stage_count": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "doc_count": 18,
                    "key": "dev"
                },
                {
                    "doc_count": 2,
                    "key": "prod"
                }
            ]
        }
    }
}

Now I want to create two triggers to send an alert to 2 different slack channels:
First trigger should be fired if there is doc_count > 0 and key = “prod”
Second trigger should be fired if there is doc_count > 0 and key = “dev”
I’ve tried following variant:

{
    "buckets_path": {
        "_count": "stage_count.doc_count",
        "_stage": "stage_count.key"
    },
    "parent_bucket_path": "composite_agg",
    "script": {
        "source": "params._count > 0 and params._stage = 'prod'"
    }
}

but I get only following response: Empty list doesn't contain element at index 0.

2

I found out at least how correctly to filter by doc_count. There is actually a magic transformation with underscore:
Following variant of “Trigger condition”.

{
    "buckets_path": {
        "_count": "_count"
    },
    "parent_bucket_path": "stage_count",
    "script": {
       "source": "params._count > 0",
       "lang": "painless"
    }
}

returns following response:

[
    {
        "doc_count": 3,
        "key": "prod"
    },
    {
        "doc_count": 1,
        "key": "dev"
    }
]

But how to filter by value of “key” field and get true/false is still unclear.

By the way. If instead of “default” aggregation of Visual editor composite_agg I’m triyng to use my own stage_count I’m getting following error:

[parsing_exception] Unknown key for a START_OBJECT in [H4j8fH4BFHM_5uP2nsEj]: [stage_count_filter]., with { line=1 & col=1022 }

when I’m trying to apply additional filter:

  "stage_count_filter": {
      "key": {
         "include": "dev"
      }
   }
3

Have you managed to figure it out how to access the key while iterate though the bucket?

4

Yes, I’ve got a good answer on github:
I adjusted it to my case - one trigger for all dev stages (by exclude filter):

{
    "buckets_path": {
        "doc_count": "_count"
    },
    "parent_bucket_path": "stage_count",
    "script": {
        "source": "params.doc_count > 0 ",
        "lang": "painless"
    },
    "gap_policy": "skip",
    "filter": {
        "exclude": "prod"
    }
}

and second one for prod:

{
    "buckets_path": {
        "doc_count": "_count"
    },
    "parent_bucket_path": "stage_count",
    "script": {
        "source": "params.doc_count > 0 ",
        "lang": "painless"
    },
    "gap_policy": "skip",
    "filter": {
        "include": "prod"
    }
}

and a monitor perfectly works!

5

Thank you for the reply @vzh.

I wonder how you do within the trigger. I try to access different bucket from the example you provide. however, I can’t really access that value within that bucket

6

ah okay. I kind of get how to do this.

I wonder if you try to access the details from the bucket to have the key and value to the messages

7

At the end, I am using the monitor configuration like this.
However, I am not sure how to access the bucket key and bucket doc_count without using the syntax like {{ctx.results.0.aggregations.response.buckets.1.key}} and {{ctx.results.0.aggregations.response.buckets.1.doc_count}}

After the filter, can we just access the filtered bucket to get the key and value?

PUT _plugins/_alerting/monitors/PdbUC38BMQQOgIcx73Sx
{
  "type": "monitor",
  "name": "alert-demo-888",
  "monitor_type": "bucket_level_monitor",
  "enabled": true,
  "schedule": {
    "period": {
      "interval": 1,
      "unit": "MINUTES"
    }
  },
  "inputs": [
    {
      "search": {
        "indices": [
          "opensearch_dashboards_sample_data_logs"
        ],
        "query": {
          "size": 1,
          "query": {
            "match_all": {}
          },
          "aggs": {
            "response": {
              "terms": {
                "field": "response.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  ],
  "triggers": [
    {
      "bucket_level_trigger": {
        "name": "trigger-demo-404",
        "severity": "1",
        "condition": {
          "buckets_path": {
            "_count": "_count"
          },
          "parent_bucket_path": "response",
          "script": {
            "source": """
            params._count > 10
            """,
            "lang": "painless"
          },
          "filter": {
              "include": "404"
          }
        },
        "actions": [
          {
            "name": "test-action",
            "destination_id": "3yt6-34BSFe9yDgKGlUf",
            "message_template": {
              "source": """
              {
                  "Monitor": "{{ctx.monitor.name}",
				          "Severity": "{{ctx.trigger.severity}}",
				          "Period start": "{{ctx.periodStart}}",
				          "Period end": "{{ctx.periodEnd}}",
				          "error_code": "{{ctx.results.0.aggregations.response.buckets.1.key}}",
				          "count": "{{ctx.results.0.aggregations.response.buckets.1.doc_count}} "
                  }
              """,
              "lang": "mustache"
            },
            "throttle_enabled": false,
            "throttle": {
              "value": 10,
              "unit": "MINUTES"
            },
            "action_execution_policy": {
              "action_execution_scope": {
                "per_alert": {
                  "actionable_alerts": [
                    "DEDUPED",
                    "NEW"
                  ]
                }
              }
            },
            "subject_template": {
              "source": "alert-demo-404",
              "lang": "mustache"
            }
          }
        ]
      }
    }
  ]
}

It would be more straightforward if I can access the bucket “xNZYDH8BMQQOgIcxOnYR”

{
  "monitor_name" : "alert-demo-888",
  "period_start" : 1645179524989,
  "period_end" : 1645179584989,
  "error" : null,
  "input_results" : {
    "results" : [
      {
        "_shards" : {
          "total" : 1,
          "failed" : 0,
          "successful" : 1,
          "skipped" : 0
        },
        "hits" : {
          "hits" : [
            {
              "_index" : "opensearch_dashboards_sample_data_logs",
              "_type" : "_doc",
              "_source" : {
                "referer" : "http://twitter.com/success/wendy-lawrence",
                "request" : "/opensearch/opensearch-1.0.0.deb",
                "agent" : "Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1",
                "extension" : "deb",
                "memory" : null,
                "ip" : "223.87.60.27",
                "index" : "opensearch_dashboards_sample_data_logs",
                "message" : "223.87.60.27 - - [2018-07-22T00:39:02.912Z] \"GET /opensearch/opensearch-1.0.0.deb_1 HTTP/1.1\" 200 6219 \"-\" \"Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1\"",
                "url" : "https://artifacts.opensearch.org/downloads/opensearch/opensearch-1.0.0.deb_1",
                "tags" : [
                  "success",
                  "info"
                ],
                "geo" : {
                  "srcdest" : "IN:US",
                  "src" : "IN",
                  "coordinates" : {
                    "lon" : -88.8454325,
                    "lat" : 39.41042861
                  },
                  "dest" : "US"
                },
                "utc_time" : "2022-02-06T00:39:02.912Z",
                "bytes" : 6219,
                "machine" : {
                  "os" : "win 8",
                  "ram" : 8589934592
                },
                "response" : 200,
                "clientip" : "223.87.60.27",
                "host" : "artifacts.opensearch.org",
                "event" : {
                  "dataset" : "sample_web_logs"
                },
                "phpmemory" : null,
                "timestamp" : "2022-02-06T00:39:02.912Z"
              },
              "_id" : "mSsZ-34BSFe9yDgKQR44",
              "_score" : 1.0
            }
          ],
          "total" : {
            "value" : 10000,
            "relation" : "gte"
          },
          "max_score" : 1.0
        },
        "took" : 3,
        "timed_out" : false,
        "aggregations" : {
          "xNZYDH8BMQQOgIcxOnYR" : {
            "parent_bucket_path" : "response",
            "bucket_indices" : [
              1
            ]
          },
          "response" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "doc_count" : 12832,
                "key" : "200"
              },
              {
                "doc_count" : 801,
                "key" : "404"
              },
              {
                "doc_count" : 441,
                "key" : "503"
              }
            ]
          }
        }
      }
    ],
    "error" : null
  },
  "trigger_results" : {
    "xNZYDH8BMQQOgIcxOnYR" : {
      "name" : "trigger-demo-404",
      "agg_result_buckets" : {
        "404" : {
          "agg_alert_content" : {
            "parent_bucket_path" : "response",
            "bucket_keys" : [
              "404"
            ],
            "bucket" : {
              "doc_count" : 801,
              "key" : "404"
            }
          }
        }
      },
      "action_results" : {
        "404" : {
          "xdZYDH8BMQQOgIcxOnYR" : {
            "id" : "xdZYDH8BMQQOgIcxOnYR",
            "name" : "test-action",
            "output" : {
              "subject" : "alert-demo-404",
              "message" : """
              {
                  "Monitor": "",
				          "Period start": "2022-02-18T10:18:44.989Z",
				          "Period end": "2022-02-18T10:19:44.989Z",
				          "error_code": "404",
				          "count": "801 "
                  }
              """,
              "messageId" : ""
            },
            "throttled" : false,
            "executionTime" : 1645179585511,
            "error" : null
          }
        }
      },
      "error" : null
    }
  }
}
8

Sorry here I can’t help you, I’m not so big expert in elastic.
But I’ve got one important thing. Guys who develop alerts for OpenSearch didn’t invent a wheel. They just more deeply know the Elasticsearch queries engine.
Here is documentation about Bucket aggregations / Terms aggregation
As you can see a syntax is very close to what is used in alerts.
And I would say your goal is to build very good query to get is much as possible simpler answer from a server with expected data to use simple filters and conditions.