Incorrect HTTP method trying to create Alerting destination via API

Plugins Alerting
1

The following worked for me with v1.3.2 but isn’t working in v2.0.0:

$ cat /opt/alerting/destinations/malcolm_api_loopback_destination.json
{
  "type": "custom_webhook",
  "name": "Malcolm API Loopback Webhook",
  "custom_webhook": {
    "path": "alert",
    "header_params": {
      "Content-Type": "application/json"
    },
    "scheme": "HTTP",
    "method": "POST",
    "port": 5000,
    "host": "api"
  }
}

$ curl -L --show-error -XPOST "http://opensearch:9200/_opendistro/_alerting/destinations" -H 'osd-xsrf:true' -H 'Content-type:application/json' '-d@/opt/alerting/destinations/malcolm_api_loopback_destination.json' ; echo
{"error":"Incorrect HTTP method for uri [/_opendistro/_alerting/destinations] and method [POST], allowed: [GET]","status":405}

From what I can see in the documentation for the create destination API, POST seems to be the correct method I should be using.

When I try a GET:

$ curl -L --show-error -XGET "http://opensearch:9200/_opendistro/_alerting/destinations" -H 'osd-xsrf:true' -H 'Content-type:application/json' ; echo
{"error":{"root_cause":[{"type":"alerting_exception","reason":"Configured indices are not found: [.opendistro-alerting-config]"}],"type":"alerting_exception","reason":"Configured indices are not found: [.opendistro-alerting-config]","caused_by":{"type":"exception","reason":"org.opensearch.index.IndexNotFoundException: no such index [.opendistro-alerting-config]"}},"status":404}

Which is what I would expect, actually, as the indices for .opendistro-alerting-config haven’t been created yet, as my attempt to create a destination fails. Or is there some other API I need to call first now to initialize alerting before I can create a destination?

2

I’ll try logging an issue on GitHub.

3

Actually realizing I might be looking at old documentation (pre opendistro → opensearch) so maybe I’ve just got the API URLs wrong, I’ll test that first.

1 Like
4

Thanks for sharing your issue @tlacuache - if the newer documentation doesn’t solve your issue make sure to drop another comment in here and we’ll work through it.

Nate

5

Thanks. I updated to make sure I was using the right APIs (/_plugins/_alerting/... instead of /_opendistro/_alerting/...) but I am still getting the same errors as described in my first comment.

6

Could this have anything to do with “destinations” showing up as “deprecated” in the alerting plugin UI now? Maybe I shouldn’t even be using this API?

7

So I tried out creating a notification channel instead of a destination and I apparently did so successfully. Still, I would think if the destinations feature were deprecated it should still work until it’s removed.

The other thing I’m not quite sure about: I don’t see in the documentation for channels how to specify things like the HTTP Method (seems to default to POST, which is okay for what I need, but where’s the config option?) or to specify custom Webhook headers or Query parameters. These are options that show up in the UI, but I don’t see them covered anywhere here.

8

After some more playing around and looking at the source code/test suite of the notifications repo I was able to get things created the way I wanted to and figure out the options for the configuration of the webhook notification channel (to replace alerting destinations).

If there were anything I would suggest for the opensearch team it would be:

  • flesh out the API documentation more (in my case, all of the options for the webhook config type were not documented)
  • if you’re still supporting the deprecated alerting destinations for now, I was not able to get one created via the API, FWIW
1 Like
9

@tlacuache - Thanks so much for the suggestion. Net time I get a chance I’ll be happy to file a documentation request on your behalf. If you’re feeling so inclined, you’re also more than welcome to fill out an issue in our documentation-website repo.

Thanks!

Nate

10

Hi again @tlacuache.

The team did announce the breaking change in alerting on the roadmap which talked to the breaking changes in 2.0. I want to make sure I understand the gap though. It sounds like the API documentation could have been clearer. Anything else we could have done?

11

Thanks! I should have picked up on it there, not sure how I missed it in the breaking changes. I guess that the thing that got me confused about it being removed vs. just deprecated was the fact that the Dashboards Alerting UI plugin still had the destinations controls in them (albeit with a “deprecated” warning at the top).

Besides that, the only speedbump I ran into was trying to figure out what all of the supported parameters were for the webook configuration type. The examples show the URL but I had to go digging in the source code to find the others.

12

I appreciate the quick feedback and will get the documentation added up if you haven’t already opened a ticket.

13

Hi @tlacuache,

I also want to make sure we don’t cause confusion between the “deprecated” and “removal” labels for Alerting.

The reason the Destination page in the Alerting UI says deprecated is because the Destinations are still present in a read-only state as fallbacks but upon upgrading to 2.0 (along with Notifications plugin installed) the Destinations will automatically be migrated to Notification Channels.

From the Alerting API perspective, the Destination write APIs have been removed and the read APIs are deprecated with a plan to remove them as well in 3.0. There should be references to this in the breaking changes as @jdbright mentioned and we do have some FAQ sections in the documentation that sheds some more light on this migration.

Feel free to create an issue or update an existing one you might have if you feel there is anything missing here.

1 Like