Prometheus Data Source not working with custom TLDs

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch: 2.9
Dashboard: 2.9
OS: Ubuntu 22.04
Browser: Firefox

Describe the issue:
Adding prometheus as data source to OpenSearch is not possible when
the “prometheus.uri” has a custom TLDs and therefore not being one of the usual TLDs.
I just tried to connect to Prometheus based on the following documentation:
Metrics analytics - OpenSearch documentation

Are there any workarounds?

2

Hey @hm21

I have Prometheus sending data to Opensearch, its pulling metrics from a remote Graylog server.

What I did…

POST _plugins/_query/_datasources 
{
    "name" : "graylog",
    "connector": "prometheus",
    "properties" : {
        "prometheus.uri" : "http://graylog.domain.com:9091"
    }
}

I changed prometheus port number while back.

image
image1351×220 15.2 KB

Had to wait a couple minutes and here is the results.

image
image1912×521 81.9 KB

3

Yea, this works but what if “prometheus.uri” is https://my-prometheus.local or https://my-prometheus.tech or https://my-prometheus.lan.

These don’t work. The Exception comes from the following code:

private static void validateDomain(String uriString) throws URISyntaxException {
    URI uri = new URI(uriString);
    String host = uri.getHost();
    if (host == null
        || (!(DomainValidator.getInstance().isValid(host)
            || DomainValidator.getInstance().isValidLocalTld(host)))) {
      throw new IllegalArgumentException(
          String.format("Invalid hostname in the uri: %s", uriString));
    }
  }

You can find the java file here: sql/datasources/src/main/java/org/opensearch/sql/datasources/utils/DatasourceValidationUtils.java at 6ae767cfd222bcf808a2e22c15529641bb75f999 · opensearch-project/sql (github.com)

4

Does anyone know about workarounds to get these kind of uri’s working as prometheus data source?

5

Also IP-Addresses are not valid.

6

Hey @hm21

I have a a domain .local (i.e., lab-testing.local) it will work is you have a PTR in your DNS server,If not you can edit you /etc/hosts file. Opensearch needs to find that FQDN.

192.168.1.100 my-prometheus.local my-prometheus.lan

Not sure what you error is when you try those FQDN’s beside what you post above.

7

So I need to edit the /etc/hosts file on the OpenSearch host?

8

Did you also try a domain with .lan TLD? And I don’t really get why a PTR is needed. For a domain with .com TLD a PTR is also not needed, right?

9

Hey @hm21

Simply put, Opensearch needs to find where my-prometheus.local is located.
So somewhere you need to tell Opensearch where my-prometheus.local = <some_ip_address>

Edit: TLDs that aren’t sanctioned or recognized by ICANN and only exist on their own private root DNS servers.

10

It doesn’t work when OpenSearch is deployed in docker containers

11

I just added my-prometheus.an to /etc/hosts file on the host machine and not the opensearch container but it still won’t work. :frowning:

12

Would you like to try it out in a container?

13

Hey @hm21

No im good with containers, perhaps someone else here has.

14

Why are you good with containers? xD

15

@hm21
Appologies, I meant to say Im not good.

16

oh, okay. dont worry. i will come back when i have the solution for the problem!