Unable to send any alerts from OpenDistro Kibana to Custom webhook (3rd party tool)

Plugins Alerting
1

Hi Team,

We are trying to configure alerting from OpenDistro Kibana to the 3rd party URL i.e., to Zenoss to generate alerts . I have selected the " Custom webhook " option and have provided the 3rd party URL - “https://******.com/zport/dmd/Events/evconsole_router” however when we create a monitor and trigger alert, We do not see any message going to Zenoss. I have tried sending “test message” as well however no luck.

I have attached the screenshots of the alert configuration. Please note we have checked the port and firewall and confirmed there is no issue with it. Kindly help !!

attach2
attach2739×456 39.2 KB

2

I don’t know much about Zenoss, but are you sure it wants auth as a password/username?

If it’s basic auth it’s probably wants something like key of Authorization and a value of Basic <Base64-encoded-credential-string>

3

Hi ,

Thanks much for your response.

I have tried with the format you have given still no luck.

Can someone help with an example with how the headers has to be configured. Im not finding any relatable documents for help.

basic
basic716×504 24.8 KB

4

The OpenDistro documentation on alerting does give some examples specifically, but there are a lot of variables here (is the network blocked between your node and your service, is the API endpoint correct, etc.) - it’s a troubleshooting exercise that is difficult to do remotely with no knowledge of your architecture. I would first confirm that everything is reachable from both places then start working up the complexity chain.

PS I did notice that your original screenshot is a PUT request and second is POST request.

5

Hi, Thanks for your response

Yes, I tried with both PUT and POST method to check if the test message has reached the destination but no luck.

The endpoint is defined correctly and we confirmed there is no network connectivity issues. I have tried with the template matching the 3rd party tool as well still no go.

Any examples or links you can share which can help ?

6

Can you do anything by pointing the webhook at a mock server using a similar network setup? My gut is telling me that the webhook is firing but it isn’t making it to Zenoss or it’s not formed the way Zenoss wants.

7

Hello Again

I’m entering the direct URL of Zenoss in the destination (screenshot attached).

Am I supposed to enter a custom or a unique webhook URL of zenoss instead ?

Thanks for your patience

tempsnip
tempsnip471×516 22.9 KB

8

You should be able to get more information from the logs as well. What do they say when you test the webhook from the alert.

Also, an issue that I still have is regarding the TLS cert on https://zenos… - is that self-signed?

I have an issue using a self signed cert and I don’t seem to be able to add the root CA pem cert that was used to sign it to any elastic config in a way that makes the CA be recognised.

9

Hello to debug create a test message on a monitor and run it using this endpoint

POST _opendistro/_alerting/monitors/<monitor_id>/_execute

If there is an error on the webhook url or the body of the message, this will tell you what happen errors, formatting etc, for more info