Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.5.0
Describe the issue:
I am attempting to create a monitor and set up a trigger/action with it. I am creating a per query monitor. When I select Add Trigger, the error message
‘Failed to load the destinations
[alerting_exception] all shards failed’ displays.
If i continue and create the trigger and action, when selecting 'Create Monitor" this error displays
‘Failed to create the monitor
[alerting_exception] the [enabled]
parameter can’t be updated for the object
mapping [metadata.last_run_context]’
The indexes I am querying all do this. Either single indexes, groups. In the logs I can see the indexes created.
Configuration:
Relevant Logs or Screenshots:
[root@search.x.x ~]# cat /var/log/opensearch/opensearch.log | grep shard
[2023-02-16T00:00:00,032][INFO ][o.o.c.m.MetadataCreateIndexService] [search.x.x] [sprod-audit-2023.02.16] creating index, cause [auto(bulk api)], templates , shards [1]/[1]
[2023-02-16T00:00:00,054][INFO ][o.o.c.m.MetadataCreateIndexService] [search.x.x] [sdev-audit-2023.02.16] creating index, cause [auto(bulk api)], templates , shards [1]/[1]
[2023-02-16T00:00:00,369][INFO ][o.o.c.m.MetadataCreateIndexService] [search.x.x] [vsat-rsyslog-2023.02.16] creating index, cause [auto(bulk api)], templates , shards [1]/[1]
[2023-02-16T00:00:01,813][INFO ][o.o.c.m.MetadataCreateIndexService] [search.x.x] [vsat-audit-2023.02.16] creating index, cause [auto(bulk api)], templates , shards [1]/[1]
[2023-02-16T08:43:20,506][ERROR][o.o.a.u.AlertingException] [search.x.x] Alerting error: Failed to execute phase [query], all shards failed; shardFailures {[ngKgYFiwQHSJofMutxZPSQ][.opendistro-alerting-config][0]: RemoteTransportException[[search.x.x][10.68.x.x:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [destination.name.keyword] in order to sort on]; }
org.opensearch.alerting.util.AlertingException: all shards failed
Caused by: java.lang.Exception: org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
[2023-02-16T08:44:18,234][ERROR][o.o.a.u.AlertingException] [search.x.x] Alerting error: Failed to execute phase [query], all shards failed; shardFailures {[ngKgYFiwQHSJofMutxZPSQ][.opendistro-alerting-config][0]: RemoteTransportException[[search.x.x][10.68.x.x:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [destination.name.keyword] in order to sort on]; }
org.opensearch.alerting.util.AlertingException: all shards failed
Caused by: java.lang.Exception: org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
org.opensearch.action.search.SearchPhaseExecutionException: all shards failed