If i type:
{{ctx.results.0.hits.total.value}}
On the message pane, i get “73”.
If i type:
{{ctx.results.0.hits.total.value}}
On the message pane, i get “73”.
It means that Monitor return some data. And now we should iterate over it:
{{#ctx.results.0.hits.hits}}
All data: {{_source}}
Username: {{_source.user}}
{{/ctx.results.0.hits.hits}}
Please do some tests without iteration:
-- Without iteration:
All Data: {{ctx.results.0.hits.hits.0._source}}
Timestamp: {{ctx.results.0.hits.hits.0._source.@timestamp}}
Scope: {{ctx.results.0.hits.hits.0._source.Scope}}
Message: {{ctx.results.0.hits.hits.0._source.Message}}
No results…
Just to clarify, I do have results on “Discovery” and can make alerts by specific terms (which all work).
Just prepared a short guide how to create an alarm based on the query. You can easy test it or in case of need adapt to your case.
The index with the prefix vpn-log-test
will be created. You can change prefix in the scrips variable
# Variables
elasticsearch_url=http://localhost:9200
date=$(date +%Y-%m-%d)
index_name=vpn-log-test-$date
index_type=default
users="Alice Bob"
error="VPN connection failed"
# Log to the Elasticsearch
for user in $users; do
time=$(date +%Y-%m-%d'T'%H:%M:%S.%3N)
curl -H "Content-Type: application/json" \
-XPOST "$elasticsearch_url/$index_name/$index_type" \
-d "{\"Time\":\"$time\", \"User\":\"$user\", \"Error\":\"$error\"}"
sleep 2
done
Kibana --> Management --> Index Patterns --> Create index pattern:
Index pattern: vpn-log-test*
Time Filter field name: time
--> Create index pattern
{
"size": 1000,
"query": {
"bool": {
"filter": [
{
"range": {
"Time": {
"from": "{{period_end}}||-10h",
"to": "{{period_end}}",
"include_lower": true,
"include_upper": true,
"format": "epoch_millis",
"boost": 1
}
}
},
{
"match_phrase": {
"Error": {
"query": "*VPN connection failed*",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {}
}
Message
Monitor {{ctx.monitor.name}} just entered alert status. Please investigate the issue.
- Trigger: {{ctx.trigger.name}}
- Severity: {{ctx.trigger.severity}}
- Period {{ctx.periodStart}} - {{ctx.periodEnd}}
- Count: {{ctx.results.0.hits.total.value}}
{{#ctx.results.0.hits.hits}}
{{_source.Time}} - {{_source.User}} - {{_source.Error}}
{{/ctx.results.0.hits.hits}}
Question related to the alarm based on the chart still need to be investigated.
Found an interesting information in the forum thread:
You could click the
info
button near theTrigger condition
editor area inDefine trigger
page. It will show you directly with JSON response of what’s available under the “ctx” variable.
I can see such button only in case of the query, not chart.
Wow, Thank you for elaborating.
I think I kind of did all.
I deleted the old VPN index and created a new index pattern, called “checkpoint*”. (our firewall)
The data get indexed successfully on the “discovery” node, as can be seen here (I filtered the results):
I created a monitor, based on the query for failed vpn login (shows hits), as can be seen here:
After that, I created a trigger (which also shows hits):
Created an Alarm \ Action:
Everything seems to be configured correctly.
Alerts are being sent by email without further issues everytime there’s a failed login (usually on the monitor I’m using 1 minutes and not 1 day as the example above).
The basic CTX commands are working without any issues but when I’m trying to use one of your suggestions above (with or without iteration) nothing seems to be displayed.
I’m not an experienced programmer, am i missing something during the process?
Thanks again!
Your help is much appreciated.
I did a test configuration above using the ‘query’ and ‘chart’. With the ‘query’ I got all required fileds in the messages but with ‘chart’ no, like in your case. It should be investigated why - maybe it is not supported at all.
For you case you can create a test Monitor with the following data:
{
"size": 1000,
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"from": "{{period_end}}||-10h",
"to": "{{period_end}}",
"include_lower": true,
"include_upper": true,
"format": "epoch_millis",
"boost": 1
}
}
},
{
"match_phrase": {
"product": {
"query": "*Mobile Access*",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
},
{
"match_phrase": {
"status": {
"query": "failure",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {}
}
And then create a Trigger with the alert and use ctx iteration in the message.
Hey @stmx38
Could you kindly assist me with writing a query for a monitor that is using more than two terms that one of them is “NOT” X, for example:
That is what I wrote (as far as I understand on how it should work):
{
"size": 1000,
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"from": "{{period_end}}||-1m",
"to": "{{period_end}}",
"include_lower": true,
"include_upper": true,
"format": "epoch_millis",
"boost": 1
}
}
},
{
"match_phrase": {
"product": {
"query": "*IPS*",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
},
{
"match_phrase": {
"not action": {
"query": "*drop*",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
},
{
"match_phrase": {
"dst": {
"query": "10.10.10.10",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {}
}
Is that correct?
Also, is there an easy-to-understand documentation on writing queries?
Thanks again!
Your filter in KQL looks like:
product: "*IPS*"
AND dst: “172.16.11.1” AND NOT action: “drop”`
{
"size": 1000,
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"from": "{{period_end}}||-1m",
"to": "{{period_end}}",
"include_lower": true,
"include_upper": true,
"format": "epoch_millis",
"boost": 1
}
}
},
{
"match_phrase": {
"product": {
"query": "*IPS*",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
},
{
"match_phrase": {
"dst": {
"query": "172.16.11.1",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"must_not": [
{
"match": {
"action": {
"query": "*drop*",
"prefix_length": 0,
"max_expansions": 50,
"fuzzy_transpositions": true,
"lenient": false,
"zero_terms_query": "NONE",
"auto_generate_synonyms_phrase_query": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {}
}
In the Discover we can see our current query in the full-text form inside: Inspect --> Request
:
Some additional documentation can be found here:
On the creen you provided I see 3 conditions and it looks like it is done in the process of the Monitor definition.
In my Kibana I didn’t find a way to configure more than one condition:
How did you do it?
Hey @stmx38,
Thanks!
Regarding the 3 conditions,
I just typed it in the last box, it was working and showed hits.
Regarding full-text query of:
product: “IPS”` AND dst: “x.x.x.x” AND NOT action: “drop”
If I want to write “contains” instead of “is”, what do i need to change in the code above?
Thanks again, much appreciation.
Will this work?
contains
"product": {
"query": "*IPS*"
is
"dst": {
"query": "10.10.10.10"
Hi folks.
JFYI: in discovery tab you can setup all the filters you need in Alert. Then you can click “Inspet” - “Request” copy request and modify it.
Hi there,
I know some time has now past, but I believe the issue with using the visual graph to define the query is that it sets the return size to 0 so there is no result set to iterate over. However there is no option to change this from the graph page.
If you define the query with the visual graph and then switch to the the extraction query you can alter the “size”: 0 to “size”: 1000 and you should see the result set come back in the “hits”.
It is annoying this seems to be the default behavior when using the visual method.
You can see exactly what is being stored when using the visual graph by using ```
GET _opendistro/_alerting/monitors/<monitor_id>
I know even more time has passed. But I just created an account to thank you for this post that ended hours of me pulling my hair out in attempts to solving this puzzle. Thank you so much.
I have struggled with this alert thing with moustache for months with no clear examples in documentation to guide but stumbling into this saved the months. Thanks a lot stmx38
this information is absolutely missing in the opensearch documentation
i can’t remember reading about the ctx _source
earlier when getting in touch with opensource projects it was mostly RTFM
sadly, today it looks like it’s RTFC
that’s not userfriendly
what happened
kudos @stmx38