How do i include message fields in the Alert Action Message

Hello,
I use OpenSearch for logs from docker containers.
I created an alert to Slack and would like to include logs message fields inside the Alert Action Message,
For example here is my lo JSON that i would like to take some fields from it(like container_name) and include them into Action Message to be send to slack.
Screen Shot 2022-01-02 at 16.13.25

Currently my Action Message is the default one.
Monitor {{ctx.monitor.name}} just entered alert status. Please investigate the issue.

  • Message: {{ctx.monitor.name}}
  • Trigger: {{ctx.trigger.name}}
  • Severity: {{ctx.trigger.severity}}
  • Alert time: {{ctx.periodStart}}

I tried to access fields by doing something like that but without success:

Monitor {{ctx.monitor.name}} just entered alert status. Please investigate the issue.> Alert time {{ctx.periodStart}}

Errors Count: {{ctx.results.0.hits.total.value}}
{{#ctx.results.0.hits.hits}}
{{_source.container.name}} - {{_source.host}} - {{_source.m.alert.severity}}
{{_source.m.alert.name}}
{{/ctx.results.0.hits.hits}}

How can i do that?
Thank you.

1 Like

I figured it out, thanks.

Glad to hear you figured it out @AndreyB - if you have the time to post your solution, I’m sure the rest of the community would find it useful

1 Like

So here is the solution:

  1. Discover the data you want to add alert to it in index.
  2. Save the query.
  3. Create a monitor based on query and change the default format to epoch_millis.
{
    "size": 500,
    "query": {
        "bool": {
            "filter": [
                {
                    "match_all": {
                        "boost": 1
                    }
                },
                {
                    "exists": {
                        "field": "m.alert.severity",
                        "boost": 1
                    }
                },
                {
                    "exists": {
                        "field": "m.alert.name",
                        "boost": 1
                    }
                },
                {
                    "range": {
                        "@timestamp": {
                            "from": "{{period_end}}||-2m",
                            "to": "{{period_end}}",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "epoch_millis",
                            "boost": 1
                        }
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1
        }
    },
    "version": true,
    "_source": {
        "includes": [],
        "excludes": []
    },
    "stored_fields": "*",
    "docvalue_fields": [
        {
            "field": "@timestamp",
            "format": "date_time"
        },
        {
            "field": "created",
            "format": "date_time"
        },
        {
            "field": "m.params.cartProducts.createdTime",
            "format": "date_time"
        },
        {
            "field": "m.time",
            "format": "date_time"
        }
    ],
    "script_fields": {},
    "sort": [
        {
            "@timestamp": {
                "order": "desc",
                "unmapped_type": "boolean"
            }
        }
    ],
    "aggregations": {
        "2": {
            "date_histogram": {
                "field": "@timestamp",
                "time_zone": "Asia/Jerusalem",
                "fixed_interval": "1s",
                "offset": 0,
                "order": {
                    "_key": "asc"
                },
                "keyed": false,
                "min_doc_count": 1
            }
        }
    },
    "highlight": {
        "pre_tags": [
            "@opensearch-dashboards-highlighted-field@"
        ],
        "post_tags": [
            "@/opensearch-dashboards-highlighted-field@"
        ],
        "fragment_size": 2147483647,
        "fields": {
            "*": {}
        }
    }
}
1 Like

Thank you @AndreyB - appreciate the write up!

1 Like

Hi @AndreyB,

thanks a lot.
I’ve got same challenge.
Is there a way to send PM to you?
Thanks

Hi @zubrik ,
You can write it here, we will try to figure it out.
Or jut send me a private message.

1 Like

I do what you indicated, but for some reason, when sending an email, all the required fields are empty, could you please tell me or show a complete picture of your monitor: trigger, query and action

Hi @IvanC ,
If you send a test e-mail you will not see any of the fields except the timestamp.
You need to simulate the real issue and check.

The fact is that I send the actual problem, I understand that there are no fields in the test message, but the problem persists (

Therefore, it seems to me that I indicated something incorrectly in the query or trigger itself.

Have you added the needed fields to stored_fields in a query editor?

"docvalue_fields": [
        {
            "field": "@timestamp",
            "format": "date_time"
        },
                {
            "field": "container.name",
            "format": "string"
        },
        {
            "field": "host",
            "format": "string"
        },
        {
            "field": "m.alert.severity",
            "format": "string"
        },
        {
            "field": "m.alert.name",
            "format": "string"
        },
        {
            "field": "m.stackTrace",
            "format": "string"
        }

And then you should define those field in the Action:

 Monitor {{ctx.monitor.name}} just entered ALERT status. Please investigate the issue.
- Alert time: {{ctx.periodStart}}

{{#ctx.results.0.hits.hits}}
> {{_source.container.name}} - {{_source.host}} - {{_source.m.alert.severity}}
> {{_source.m.alert.name}}
> {{_source.m.stackTrace}}
{{/ctx.results.0.hits.hits}}

@AndreyB where can i add those docvalue after aggregations or before


so i have this query and query response like this
Screenshot from 2022-02-23 09-55-32
so now i want to get that query in email alert how can i do that help me to figure it out thanks.

In my code it comes after the aggregation.
Take a look

{
    "size": 500,
    "query": {
        "bool": {
            "filter": [
                {
                    "match_all": {
                        "boost": 1
                    }
                },
                {
                    "exists": {
                        "field": "m.alert.severity",
                        "boost": 1
                    }
                },
                {
                    "exists": {
                        "field": "m.alert.name",
                        "boost": 1
                    }
                },
                {
                    "range": {
                        "@timestamp": {
                            "from": "{{period_end}}||-2m",
                            "to": "{{period_end}}",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "epoch_millis",
                            "boost": 1
                        }
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1
        }
    },
    "version": true,
    "_source": {
        "includes": [],
        "excludes": []
    },
    "stored_fields": "*",
    "docvalue_fields": [
        {
            "field": "@timestamp",
            "format": "date_time"
        },
        {
            "field": "created",
            "format": "date_time"
        },
        {
            "field": "m.params.cartProducts.createdTime",
            "format": "date_time"
        },
        {
            "field": "m.time",
            "format": "date_time"
        }
    ],
    "script_fields": {},
    "sort": [
        {
            "@timestamp": {
                "order": "desc",
                "unmapped_type": "boolean"
            }
        }
    ],
    "aggregations": {
        "2": {
            "date_histogram": {
                "field": "@timestamp",
                "time_zone": "Asia/Jerusalem",
                "fixed_interval": "1s",
                "offset": 0,
                "order": {
                    "_key": "asc"
                },
                "keyed": false,
                "min_doc_count": 1
            }
        }
    },
    "highlight": {
        "pre_tags": [
            "@opensearch-dashboards-highlighted-field@"
        ],
        "post_tags": [
            "@/opensearch-dashboards-highlighted-field@"
        ],
        "fragment_size": 2147483647,
        "fields": {
            "*": {}
        }
    }
}

what are the m.alert.severity and m.alert.name those are the feilds existed in your index or what could u plz expain in detail

@sateesh m.alert its just a field that i’m creating using my logstash parsing.
It can be any field in you case, it’s just a name.

okay but here in my case we are not using logstash directly we are pushing metrics to es using metricbeat. same like your script i just wrote
{
“size”: 500,
“query”: {
“bool”: {
“filter”: [
{
“match_all”: {
“boost”: 1
}
},
{
“exists”: {
“field”: “mysql.slowlog.query”,
“boost”: 1
}
},
{
“exists”: {
“field”: “event.duration”,
“boost”: 1
}
},
{
“range”: {
@timestamp”: {
“from”: “{{period_end}}||-5m”,
“to”: “{{period_end}}”,
“include_lower”: true,
“include_upper”: true,
“format”: “epoch_millis”,
“boost”: 1
}
}
}
],
“adjust_pure_negative”: true,
“boost”: 1
}
},
“version”: true,
“_source”: {
“includes”: ,
“excludes”:
},
“stored_fields”: “*”,
“docvalue_fields”: [
{
“field”: “@timestamp”,
“format”: “date_time”
},
{
“field”: “mysql.slowlog.query”,
“format”: “string”
}

],
"aggregations": {
     "display": {
        "terms": {
            "field": "mysql.slowlog.query",
            "size": 10,
            "min_doc_count": 1,
            "shard_min_doc_count": 0,
            "show_term_doc_count_error": false,
            "order": [
                {
                    "_count": "desc"
                },
                {
                    "_key": "asc"
                }
            ]
        }
    }
}

    }

but i am unable to get those feilds, do i mis anything here in script i am gettinf the results but unable get those action

@sateesh Where did you get this script from?
I will tell you what i do.

  1. Filter the data i want to add alert to it in index in Discover tab.
  2. Press on “Inspect” on the upper right corner and press “Request”.
  3. Copy the script. This is the script i create monitor based on it and this script already have “docvalue_fields”.

I tried the same way the response is different when i copy that script to monitor i don’t know why ,which version r u using ?? i am using elasticsearch 7.10.2

@sateesh i’m using OpenSearch 1.2.4.