Prometheus does not scrape from prometheus-exporter-plugin-for-opensearch

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch v.12.1.0
Plugin Version v.12.1.0
Ubuntu 20.04

Describe the issue:
I have installed opensearch plugin prometheus-exporter-plugin-for-opensearch
I can see metrics from curl but prometheus does not scrape metrics.
Is there anything with my configuration or permissions?

Configuration:
Prometheus.yml

  • job_name: opensearch
    scrape_interval: 15s
    scheme: https
    tls_config:
    insecure_skip_verify: true
    metrics_path: “/_prometheus/metrics”
    static_configs:
    • targets: [‘user:passwd@localhost:9200’]

I have added these permissions also to plugin-security.policy
permission java.lang.RuntimePermission “accessClassInPackage.sun.misc”;

permission java.lang.RuntimePermission “accessDeclaredMembers”;

permission java.lang.reflect.ReflectPermission “suppressAccessChecks”;

Relevant Logs or Screenshots:

image
image714×644 32.3 KB

2

Hey @omers

By chance is that v2.1.0?

If Prometheus is in the same node as your OS/OSD I think you can just use localhost.
Also By default it uses port 9090.
Something like this works.

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

Then on the Web UI configure your Data Source.

POST _plugins/_query/_datasources 
{
    "name" : "my_prometheus",
    "connector": "prometheus",
    "properties" : {
        "prometheus.uri" : "http://localhost:9090"
    }
3

Hi @Gsmitt ,
Sorry I mistyped the version it should be v.2.11.1
Yes prometheus is on the same vm with OS.
Theere is already config for prometheus and metrics for CPU RAM etc. can be seen.
I want to reach metrics for OS like query rate, index rate like the below (I can see from browser :9200/_prometheus/metrics)

image
image878×422 8.47 KB

Then see these from Grafana dashboard which shows No Data at the moment.

4

Hey @omers

Unless you have changed the port on OpenSearch (by default it uses port 9200). With Prometheus try using a different port like 9090 or 9091 instead of 9200. See if that works.

5

Hi @Gsmitt
Actually I could not find how to change the plugin port from 9200 to other port at the moment, i look for it
But catch a glue on the error, it happens because of config of target (@ and colon characters)
I will write here if i can work around that
Thanks

image
image1007×575 47.6 KB

6

Hey @omers

Looking at your screenshot, it looks correct but your configurations for Prometheus configuration file is using OpenSearch default port( 9200) I can see it shows invalid URL and “too many colons in address”

Try change this section.

Perhaps try this.

* targets: [‘user:passwd@localhost:9090’]