Error: Failed to run the trigger [parsing_exception]

divyank_1 · December 14, 2022, 2:45pm

I am using Opensearch version: 2.0.0

While creating monitor using bucket_level_monitor, the extraction query editor in trigger condition giving error.

Error:
Failed to run the trigger [parsing_exception] Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT], with { line=1 & col=388 }Failed to run the trigger [parsing_exception] Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT], with { line=1 & col=388 }

Snap:

Getting parsing exception while testing trigger condition but alerts are getting triggered in per bucket monitor.

I changed trigger name as Alerts fromtrigger name : hi to Doc count > 0 as alerts are not getting stopped.

Now for new trigger name alerts are not getting triggered.

These alerts are generated by using composite aggregations.

qreshi · December 14, 2022, 6:18pm

Hi @divyank_1,

Would you mind fetching the full stacktrace by searching for this error in the backend opensearch.log file and providing that here?

divyank_1 · December 15, 2022, 5:15am

Hi @qreshi,
Pls find logs:

opensearch-dashboards    | Alerting - MonitorService - executeMonitor: StatusCodeError: [parsing_exception] Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT], with { line=1 & col=388 }
opensearch-dashboards    |     at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
opensearch-dashboards    |     at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
opensearch-dashboards    |     at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
opensearch-dashboards    |     at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
opensearch-dashboards    |     at IncomingMessage.emit (events.js:412:35)
opensearch-dashboards    |     at endReadableNT (internal/streams/readable.js:1334:12)
opensearch-dashboards    |     at processTicksAndRejections (internal/process/task_queues.js:82:21) {
opensearch-dashboards    |   status: 400,
opensearch-dashboards    |   displayName: 'BadRequest',
opensearch-dashboards    |   path: '/_plugins/_alerting/monitors/_execute?dryrun=true',
opensearch-dashboards    |   query: {},
opensearch-dashboards    |   body: {
opensearch-dashboards    |     error: {
opensearch-dashboards    |       root_cause: [Array],
opensearch-dashboards    |       type: 'parsing_exception',
opensearch-dashboards    |       reason: 'Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT]',
opensearch-dashboards    |       line: 1,
opensearch-dashboards    |       col: 388
opensearch-dashboards    |     },
opensearch-dashboards    |     status: 400
opensearch-dashboards    |   },
opensearch-dashboards    |   statusCode: 400,
opensearch-dashboards    |   response: '{"error":{"root_cause":[{"type":"parsing_exception","reason":"Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT]","line":1,"col":388}],"type":"parsing_exception","reason":"Failed to parse object: expecting token of type [START_OBJECT] but found [END_OBJECT]","line":1,"col":388},"status":400}',
opensearch-dashboards    |   toString: [Function (anonymous)],
opensearch-dashboards    |   toJSON: [Function (anonymous)]
opensearch-dashboards    | }
opensearch-dashboards    | {"type":"response","@timestamp":"2022-12-15T05:18:58Z","tags":[],"pid":1,"method":"post","statusCode":200,"req":{"url":"/api/alerting/monitors/_execute","method":"post","headers":{"host":"localhost:5601","connection":"keep-alive","content-length":"1208","sec-ch-ua":"\"Not?A_Brand\";v=\"8\", \"Chromium\";v=\"108\", \"Brave\";v=\"108\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36","osd-version":"2.0.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-gpc":"1","accept-language":"en-US,en","origin":"http://localhost:5601","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"http://localhost:5601/app/alerting","accept-encoding":"gzip, deflate, br","securitytenant":"__user__"},"remoteAddress":"172.21.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36","referer":"http://localhost:5601/app/alerting"},"res":{"statusCode":200,"responseTime":26,"contentLength":9},"message":"POST /api/alerting/monitors/_execute 200 26ms - 9.0B"}
opensearch-node1         | [2022-12-15T05:19:35,287][INFO ][o.o.a.InputService       ] [opensearch-node1] Error collecting inputs for monitor: _4K5EIUBKQlNyuLtsKGI
opensearch-node1         | org.opensearch.action.ActionRequestValidationException: Validation Failed: 1: No aggregation found for path [container];
opensearch-node1         |      at org.opensearch.action.ValidateActions.addValidationError(ValidateActions.java:44) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.search.aggregations.PipelineAggregationBuilder$ValidationContext.addValidationError(PipelineAggregationBuilder.java:242) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.search.aggregations.AggregatorFactories$Builder.validatePipelines(AggregatorFactories.java:388) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.search.aggregations.AggregatorFactories$Builder.validate(AggregatorFactories.java:376) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.action.search.SearchRequest.validate(SearchRequest.java:328) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.action.support.TransportAction.execute(TransportAction.java:163) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.action.support.TransportAction.execute(TransportAction.java:102) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:423) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.client.support.AbstractClient.search(AbstractClient.java:554) ~[opensearch-2.0.0.jar:2.0.0]
opensearch-node1         |      at org.opensearch.alerting.InputService$collectInputResults$2$searchResponse$1.invoke(InputService.kt:83) ~[opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.InputService$collectInputResults$2$searchResponse$1.invoke(InputService.kt:83) ~[opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.opensearchapi.OpenSearchExtensionsKt.suspendUntil(OpenSearchExtensions.kt:189) ~[alerting-core-2.0.0.0.jar:?]
opensearch-node1         |      at org.opensearch.alerting.InputService.collectInputResults(InputService.kt:83) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$2.invokeSuspend(BucketLevelMonitorRunner.kt:88) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$2.invoke(BucketLevelMonitorRunner.kt) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$2.invoke(BucketLevelMonitorRunner.kt) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.opensearchapi.OpenSearchExtensionsKt$withClosableContext$2.invokeSuspend(OpenSearchExtensions.kt:253) [alerting-core-2.0.0.0.jar:?]
opensearch-node1         |      at org.opensearch.alerting.opensearchapi.OpenSearchExtensionsKt$withClosableContext$2.invoke(OpenSearchExtensions.kt) [alerting-core-2.0.0.0.jar:?]
opensearch-node1         |      at org.opensearch.alerting.opensearchapi.OpenSearchExtensionsKt$withClosableContext$2.invoke(OpenSearchExtensions.kt) [alerting-core-2.0.0.0.jar:?]
opensearch-node1         |      at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:146) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.BuildersKt.withContext(Unknown Source) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at org.opensearch.alerting.opensearchapi.OpenSearchExtensionsKt.withClosableContext(OpenSearchExtensions.kt:253) [alerting-core-2.0.0.0.jar:?]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner.runMonitor(BucketLevelMonitorRunner.kt:83) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$1.invokeSuspend(BucketLevelMonitorRunner.kt) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         | [2022-12-15T05:19:35,289][INFO ][o.o.a.TriggerService     ] [opensearch-node1] Error running trigger [ioLCEIUBKQlNyuLtC6L-] for monitor [_4K5EIUBKQlNyuLtsKGI]
opensearch-node1         | java.lang.IndexOutOfBoundsException: Empty list doesn't contain element at index 0.
opensearch-node1         |      at kotlin.collections.EmptyList.get(Collections.kt:36) ~[kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at kotlin.collections.EmptyList.get(Collections.kt:24) ~[kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at org.opensearch.alerting.TriggerService.runBucketLevelTrigger(TriggerService.kt:96) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner.runMonitor(BucketLevelMonitorRunner.kt:106) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$1.invokeSuspend(BucketLevelMonitorRunner.kt) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         | [2022-12-15T05:19:35,290][INFO ][o.o.a.TriggerService     ] [opensearch-node1] Error running trigger [jIISEYUBKQlNyuLtlqp-] for monitor [_4K5EIUBKQlNyuLtsKGI]
opensearch-node1         | java.lang.IndexOutOfBoundsException: Empty list doesn't contain element at index 0.
opensearch-node1         |      at kotlin.collections.EmptyList.get(Collections.kt:36) ~[kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at kotlin.collections.EmptyList.get(Collections.kt:24) ~[kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at org.opensearch.alerting.TriggerService.runBucketLevelTrigger(TriggerService.kt:96) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner.runMonitor(BucketLevelMonitorRunner.kt:106) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at org.opensearch.alerting.BucketLevelMonitorRunner$runMonitor$1.invokeSuspend(BucketLevelMonitorRunner.kt) [opensearch-alerting-2.0.0.0.jar:2.0.0.0]
opensearch-node1         |      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.6.10.jar:1.6.10-release-923(1.6.10)]
opensearch-node1         |      at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60) [kotlinx-coroutines-core-1.1.1.jar:?]
opensearch-node1         |      at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742) [kotlinx-coroutines-core-1.1.1.jar:?]

divyank_1 · December 15, 2022, 5:39am

Update on Above:

I created new monitor from scratch, Pls find my exact per bucket monitor query.
I am using compose aggs now.

Monitor query:

{
   "name": "per_buc_monitor-composite aggs",
   "type": "monitor",
   "monitor_type": "bucket_level_monitor",
   "enabled": true,
   "schedule": {
      "period": {
         "unit": "MINUTES",
         "interval": 1
      }
   },
   "inputs": [
      {
         "search": {
            "indices": [
               "data"
            ],
            "query": {
               "size": 0,
               "aggregations": {
                  "composite_agg": {
                     "composite": {
                        "size": 10,
                        "sources": [
                           {
                              "container": {
                                 "terms": {
                                    "field": "tag.name.keyword",
                                    "missing_bucket": false,
                                    "order": "desc"
                                 }
                              }
                           }
                        ]
                     }
                  }
               }
            }
         }
      }
   ],
   "triggers": [
      {
         "bucket_level_trigger": {
            "id": "FCJEFIUBwmh188QS6QwG",
            "name": "doc count > 0",
            "severity": "1",
            "condition": {
               "buckets_path": {
                  "count_var": "_count"
               },
               "parent_bucket_path": "composite_agg",
               "script": {
                  "source": "params.count_var > 0",
                  "lang": "painless"
               },
               "gap_policy": "skip"
            },
            "actions": []
         }
      }
   ],
   "ui_metadata": {
      "schedule": {
         "timezone": null,
         "frequency": "interval",
         "period": {
            "unit": "MINUTES",
            "interval": 1
         },
         "daily": 0,
         "weekly": {
            "tue": false,
            "wed": false,
            "thur": false,
            "sat": false,
            "fri": false,
            "mon": false,
            "sun": false
         },
         "monthly": {
            "type": "day",
            "day": 1
         },
         "cronExpression": "0 */1 * * *"
      },
      "monitor_type": "bucket_level_monitor",
      "search": {
         "searchType": "query",
         "timeField": "",
         "aggregations": [],
         "groupBy": [],
         "bucketValue": 1,
         "bucketUnitOfTime": "h",
         "where": {
            "fieldName": [],
            "fieldRangeEnd": 0,
            "fieldRangeStart": 0,
            "fieldValue": "",
            "operator": "is"
         }
      }
   }
}

When I was testing trigger condition, it’s giving error, but after creation of trigger, the alerts are working.

Snap:

Some query:

Do we need to use composite_aggs or we can use normal bucket aggs query as well.
I tried normal bucket aggs query yesterday, but it did not work.

Ref link: How to get aggregations's value？ · Issue #253 · opensearch-project/alerting · GitHub

Topic		Replies	Views
Error log - don't understand how to read it Alerting	1	357	November 8, 2022
[Error] Failed to load destinations, Failed to create monitor Alerting troubleshoot	2	842	April 17, 2023
2 configured triggers in Alert, but only 1 trigger is getting triggered Alerting	1	477	October 26, 2021
Unable to get alert - getting Error and Deleted state Security Analytics discuss	0	93	March 1, 2024
Failed to start Document-level-monitor: Inconsistency of field data structures across documents for field Security Analytics	4	561	May 28, 2023

Error: Failed to run the trigger [parsing_exception]

Related Topics