MHR
July 6, 2025, 6:16pm
1
I’m working with OpenSearch version 2.17.1 and trying to set up a composite monitor (workflow)
On the OpenSearch Dashboards UI, it’s currently not possible to add a scripted condition (painless) for a trigger — only basic logical conditions (like A AND B) are available.
To work around this, I created the trigger via the Dev Tools console with a custom condition like this:
“condition”: {
“script”: {
“source”: “return true;”,
“lang”: “painless”
}
}
However, when I try to execute the workflow, I always get an error:
Error while processing the trigger expression: returntrue;
Even with this extremely basic script, the trigger fails to execute.
I’ve checked:
• Both input monitors return valid results with hits.hits
• Monitor IDs are correct in the composite input
• The script syntax is valid
• I’ve tried escaping and formatting in various ways (using “”" or \t, etc.)
It seems like the trigger script is either not parsed correctly, or the composite monitor cannot access ctx.monitors[…] during execution.
Has anyone experienced this? Any idea how to get around it?
Thanks in advance!
pablo
July 7, 2025, 5:26pm
2
@MHR Did you follow this OpenSearch documentation?
MHR
July 8, 2025, 12:48pm
3
Hello Pablo,
Thanks for your answer. Yes, i did. I have no issue when i use a simple condition like this:
"condition": {
"script": {
"source": "(monitor[id=grsbCIcBvEHfkjWFeCqb] && monitor[id=agasbCIcBvEHfkjWFeCqa])",
"lang": "painless"
}
}
The probleme appears when i use another script, even something as simple as return true;
pablo
July 8, 2025, 9:21pm
4
@MHR According to the RFC and OpenSearch documentation, the condition only checks the alert status of each defined monitor. Then, based on the defined Boolean logic composite alert is triggered.
opened 11:18PM - 12 Dec 22 UTC
closed 02:57PM - 24 Oct 23 UTC
enhancement
### Purpose
This issue talks about chaining the Monitors in the Alerting plug… in - how to re-use the findings and alerts generated by one monitor to another monitor. It highlights the need for a new kind of monitor to handle more elaborate use cases and complex queries which are not catered to by the current implementation of Alerting and proposes a solution to achieve that.
### Background
Alerting plugin is used to monitor user data and create alerts that trigger when conditions occur in one or more indexes. Monitors are used to create triggers. Bucket level monitors create triggers which have conditions that alert on data aggregated into buckets. Query level monitors support triggers which have conditions that generate alerts on query criteria. Document level monitor can create triggers that alert on individual document matching query conditions. Each monitor performs a very specific and limited function.
* Users of alerting plugin have complex queries which are not being catered to by a single monitor and need to use a combination of triggers provided by different types of monitors.
* Users need a single view of data source during the monitor execution run to be fanned out across multiple monitors, to attain finer consistency of results. (In the current implementation of alerting plugin we cannot achieve this as all monitors run as individual jobs and we can't guarantee they are looking at the same data set)
* If monitors are chained the findings generated by one monitor can be used as the source data for a second monitor that can do further granular analysis and generate more meaningful of alerts.
* Users need the ability to define pipelines of rules and queries that will run as single execution, internally running a sequence of monitors and generating a chain of alerts.
To achieve the above vision we would need to have a more abstract way to monitor data - *composite monitors*. *Composite* monitors allow users to execute delegate monitors in sequential fashion hence providing more control over the data seen by composite monitors which will enable creation of more granular findings and chained alerts.
### Challenges Solved By Composite Monitors
### Chained Findings
Let’s consider the following example where a composite monitor allows user to chain findings to extract more value from monitor analyses and create more meaningful alerts.:
* User creates Monitor 1, a Document level monitor on a set of logs indices with the following queries - 1. Request payload size >100 kb. 2. Response Status != 200 3. Response contains a specific header. On every execution, monitor would query and generate findings mentioning which documents matched how many queries.
* Although individually each of the above 3 queries is a data metric for the user, user deems that any documents matching all 3 queries would require further analysis - say, he wants to aggregate these logs by client IP and check how many such IPs are there and how many such requests are being fired from these IPs. These cannot be catered by Monitor 1 itself and would require further processing.
* User creates Monitor 2, a bucket level monitor which analyses the findings of above mentioned document level monitor and create triggers and alert notifications for said conditions.
* User creates a composite monitor which accepts a sequence of monitors to execute. User defines the sequence to execute monitor1 followed by monitor 2, which accepts a config to use Monitor 1 findings as its data set.
* This would let user define condition to execute a monitor based on another monitor’s findings.
<img width="1218" alt="Screen Shot 2022-12-09 at 11 29 52 AM" src="https://user-images.githubusercontent.com/6163412/207169737-7111fb6e-e194-4c03-bf73-214b10aa6cbe.png">
### Chained Alerts
Let’s consider the following use case:
* User creates Monitor 1, to monitor CPU Utilization of their service’s worker nodes. Monitor has trigger conditions to create alerts on high CPU threshold. But high CPU maybe due to background processes, high traffic, cluster instabilities etc. Simply notifying on high CPU is not of much value to user.
* User creates Monitor 2, to monitor incoming request count in the same time window. Monitor has trigger conditions to create alerts on high request count threshold. But if the cluster is able to handle the heavy traffic smoothly, there is no value in creating notifications.
* User creates a composite monitor with a sequence to run monitor 1 and monitor 2 and a monitor 3. Monitor 3 accepts a config to read alerts created by both monitor 1 and monitor 2 in the current execution and if yes, create an alert and send out a notification that their service is degrading due to high traffic. This alert would help customer scale their service and serves as a more meaningful and critical alert.
<img width="808" alt="Screen Shot 2022-12-07 at 11 32 08 AM" src="https://user-images.githubusercontent.com/6163412/207179201-8cd6409c-8245-444c-8769-8f0971fc40bf.png">
### Running multiple monitors in one execution
User simply wants to run multiple monitors as a part of a composite monitor at once to ensure all the monitors are deterministically querying data in the same time range and control the sequence of execution of monitors. This also ensures that an entire workflow or pipeline created by users is solved by a single composite monitor.
<img width="787" alt="sequence of monitors" src="https://user-images.githubusercontent.com/6163412/207179922-781987e3-ef16-4847-b3a2-9fc305788d95.png">
I can’t find an option to define a custom script.