No data for any monitor query

Plugins Alerting
1

Hi,

I am running elasticsearch-oss 7.1.1, opendistroforelasticsearch 1.1.0-1, opendistro-alerting 1.1.0.0-1, and opendistroforelasticsearch-kibana 1.1.0. We started using opendistro for our prod logging and want to implement some alerts. I have been trying to configure our first “test” alert and I am running into an issue while creating the monitor. I have not been able to get any data whatsoever when configuring the monitor but will always receive “There is no data for the current selections.”. I’ve tried using both extraction query and the visual graph method and am using the admin user as well. One example is using a visual graph with a wildcard index field, @timestamp for the time field, and then count, all documents, from the last 1 hour, where kubernetes.namespace is ingress-nginx. We are constantly receiving logs in that namespace and can view the logs in the discover tab so this should show up with data. Any help or suggestions on what to do next would be appreciated

2

Hi @astrozyk,

Since you said the following:

am using the admin user as well

I assume you are using the alerting plugin together with the security plugin. This requires a bit more setup. In order to avoid alerting have more access than a user creating alerts we have limited the configuration indices to be of admin only. Unfortunately this led to an issue of using the two plugins together as the index will always be empty for our scheduler.

For now there are work arounds but they are complicated, just know that we are aware of this issue and working on a secure and safe resolution to have these plugins work together.

Monitors will run as long as the node does not go offline (a inconvenience for sure). I recommend creating a extraction query monitor looking for the fields you mentioned, this should work, and you can try something as basic as:

return ctx.results[0].hits.total.value > 0;

for your trigger condition.

If this still does not work I am going to need some more information regarding your system, and log outputs for the period of where the job gets scheduled to the end of the interval period, and .yml config (please remove any admin creds / pem key names).

We do have a github issue to add more logging information here: More logging · Issue #77 · opendistro-for-elasticsearch/alerting · GitHub feel free to :+1: it :slight_smile: .