Prometheus Data Source not working with custom TLDs

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch: 2.9
Dashboard: 2.9
OS: Ubuntu 22.04
Browser: Firefox

Describe the issue:
Adding prometheus as data source to OpenSearch is not possible when
the “prometheus.uri” has a custom TLDs and therefore not being one of the usual TLDs.
I just tried to connect to Prometheus based on the following documentation:
Metrics analytics - OpenSearch documentation

Are there any workarounds?

Hey @hm21

I have Prometheus sending data to Opensearch, its pulling metrics from a remote Graylog server.

What I did…

POST _plugins/_query/_datasources 
    "name" : "graylog",
    "connector": "prometheus",
    "properties" : {
        "prometheus.uri" : ""

I changed prometheus port number while back.

Had to wait a couple minutes and here is the results.

Yea, this works but what if “prometheus.uri” is https://my-prometheus.local or or https://my-prometheus.lan.

These don’t work. The Exception comes from the following code:

private static void validateDomain(String uriString) throws URISyntaxException {
    URI uri = new URI(uriString);
    String host = uri.getHost();
    if (host == null
        || (!(DomainValidator.getInstance().isValid(host)
            || DomainValidator.getInstance().isValidLocalTld(host)))) {
      throw new IllegalArgumentException(
          String.format("Invalid hostname in the uri: %s", uriString));

You can find the java file here: sql/datasources/src/main/java/org/opensearch/sql/datasources/utils/ at 6ae767cfd222bcf808a2e22c15529641bb75f999 · opensearch-project/sql (

Does anyone know about workarounds to get these kind of uri’s working as prometheus data source?

Also IP-Addresses are not valid.

Hey @hm21

I have a a domain .local (i.e., lab-testing.local) it will work is you have a PTR in your DNS server,If not you can edit you /etc/hosts file. Opensearch needs to find that FQDN. my-prometheus.local my-prometheus.lan

Not sure what you error is when you try those FQDN’s beside what you post above.

So I need to edit the /etc/hosts file on the OpenSearch host?

Did you also try a domain with .lan TLD? And I don’t really get why a PTR is needed. For a domain with .com TLD a PTR is also not needed, right?

Hey @hm21

Simply put, Opensearch needs to find where my-prometheus.local is located.
So somewhere you need to tell Opensearch where my-prometheus.local = <some_ip_address>

Edit: TLDs that aren’t sanctioned or recognized by ICANN and only exist on their own private root DNS servers.

It doesn’t work when OpenSearch is deployed in docker containers

I just added to /etc/hosts file on the host machine and not the opensearch container but it still won’t work. :frowning:

Would you like to try it out in a container?

Hey @hm21

No im good with containers, perhaps someone else here has.

Why are you good with containers? xD

Appologies, I meant to say Im not good.

oh, okay. dont worry. i will come back when i have the solution for the problem!