Add Exporter Plugin to Opensearch

Hello all,

I search to install opensearch exporter to collect opensearch metrics.
Compared to the plugin on open distro/elasticsearch, which was installed in standalone process

Now the Opensearch exporter is installed in plugin mode.

So i retrieve package under : GitHub - vvanholl/elasticsearch-prometheus-exporter: Prometheus exporter plugin for Elasticsearch

When i lauch install on my master opensearch container : ./opensearch-plugin install -b opensearch-prometheus-exporter-master.zip -v

Error appear : ERROR: Unknown plugin opensearch-prometheus-exporter-master.zip.

Do you have encountered this problem ? Maybe i can install it in stand alone process ?

Thanks you for yoru help.

Best Regards

Gnarly

It doesn’t look like vvanholl/elasticsearch-prometheus-exporter supports OpenSearch. Plugins have to be ported over and it doesn’t look like vvanholl has done this for that particular project.

There is a fork of this plugin which should work with OpenSearch GitHub - aparo/opensearch-prometheus-exporter: OpenSearch Prometheus Exporter (fork of https://github.com/vvanholl/elasticsearch-prometheus-exporter)

4 Likes

Great callout @jmau2002 I totally forgot about the work @aparo did on this!

Yes, thanks for sharing.

But on my pod, i tested to install this plugin directly on my pod master:

[opensearch@master-esga2-0 ~]$ ./bin/opensearch-plugin install -b bin/opensearch-prometheus-exporter-master.zip
-> Installing bin/opensearch-prometheus-exporter-master.zip
-> Failed installing bin/opensearch-prometheus-exporter-master.zip
-> Rolling back bin/opensearch-prometheus-exporter-master.zip
-> Rolled back bin/opensearch-prometheus-exporter-master.zip
A tool for managing installed opensearch plugins

Non-option arguments:
command

Option             Description
------             -----------
-E <KeyValuePair>  Configure a setting
-h, --help         Show help
-s, --silent       Show minimal output
-v, --verbose      Show verbose output
ERROR: Unknown plugin bin/opensearch-prometheus-exporter-master.zip

Maybe i miss something ? Environment (Kubernetes).

I can’t get it to work. On Linux tarball installation, the plugin installation don’t work

Hi,

I think this install instructions are dated, this is what worked for Elasticsearch in the past but now with OpenSearch you need to use:

./bin/opensearch-plugin install https://github.com/<your_path>/<plugin>.zip

Of course you can replace the GitHub URL with any valid URI that points to your local file.

Let me know if that worked for you.

Regards,
Lukáš

@GnarlyCapricorne in addition to @lukas-vlcek suggestion, if you need / want / prefer to install the plugin from the local filesystem, use file:// + full path URL notation, for example:

./bin/opensearch-plugin   install -b file:///folder/opensearch-prometheus-exporter-master.zip

Hope it helps.

Hello,

Thanks for the file://

It s always KO but the new message error is encouraging.

-> Installing file:///images/opensearch_exporter.zip
-> Downloading file:///images/opensearch_exporter.zip
Retrieving zip from file:///images/opensearch_exporter.zip
-> Failed installing file:///images/opensearch_exporter.zip
-> Rolling back file:///images/opensearch_exporter.zip
-> Rolled back file:///images/opensearch_exporter.zip
Exception in thread "main" java.nio.file.NoSuchFileException: /opt/opensearch/90/current/plugins/.installing-16875165838775592868/plugin-descriptor.properties

ZIP do not have plugin descriptor too. (on dl from git official)

I have Opensearch 1.2.3. And version of Prometheu Opensearch exporter 1.2.3 (GitHub - aparo/opensearch-prometheus-exporter: OpenSearch Prometheus Exporter (fork of https://github.com/vvanholl/elasticsearch-prometheus-exporter))

Interesting, I just did these steps on the fresh Opensearch 1.2.3 container instance:

[opensearch@ebc03b3e5ff2 ~]$ curl -L https://github.com/aparo/opensearch-prometheus-exporter/releases/download/1.2.3/prometheus-exporter-1.2.3.zip --output prometheus-exporter-1.2.3.zip
[opensearch@ebc03b3e5ff2 ~]$ ./bin/opensearch-plugin install file:///usr/share/opensearch/prometheus-exporter-1.2.3.zip 
-> Installing file:///usr/share/opensearch/prometheus-exporter-1.2.3.zip
-> Downloading file:///usr/share/opensearch/prometheus-exporter-1.2.3.zip
[=================================================] 100%?? 
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.reflect.ReflectPermission suppressAccessChecks
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed prometheus-exporter with folder name prometheus-exporter

Could you please check the size / content of your ZIP?

Hii,

I don’t know if it can help :

ls -ltrh /images/opensearch-prometheus-exporter-master.zip
-rw-r--r-- 1 root root 71K Feb  2 15:28 /images/opensearch-prometheus-exporter-master.zip

zipinfo -l /images/opensearch-prometheus-exporter-master.zip
Archive:  /images/opensearch-prometheus-exporter-master.zip
Zip file size: 72412 bytes, number of entries: 83
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/.github/
-rw----     0.0 fat       25 tx       25 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/.github/CODEOWNERS
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/.github/workflows/
-rw----     0.0 fat     1333 tx      533 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/.github/workflows/ci.yml
-rw----     0.0 fat      401 tx      270 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/.gitignore
-rw----     0.0 fat    11358 tx     3949 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/LICENSE.txt
-rw----     0.0 fat      119 tx      100 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/NOTICE.txt
-rw----     0.0 fat     4651 tx     1885 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/README.md
-rw----     0.0 fat     4681 tx     1873 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/build.gradle
-rw----     0.0 fat    12916 tx     3039 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/checkstyle.xml
-rw----     0.0 fat      317 tx      165 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/gradle.properties
-rw----     0.0 fat       41 tx       41 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/settings.gradle
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/prometheus/
-rw----     0.0 fat     5295 tx     1348 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/prometheus/PrometheusMetricsCatalog.java
-rw----     0.0 fat    68195 tx     8411 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/prometheus/PrometheusMetricsCollector.java
-rw----     0.0 fat     2802 tx     1085 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/prometheus/PrometheusSettings.java
-rw----     0.0 fat      731 tx      450 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/compuscene/metrics/prometheus/package-info.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/
-rw----     0.0 fat     7897 tx     2562 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/ClusterStatsData.java
-rw----     0.0 fat     1079 tx      571 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/NodePrometheusMetricsAction.java
-rw----     0.0 fat     1234 tx      619 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/NodePrometheusMetricsRequest.java
-rw----     0.0 fat     3264 tx      974 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/NodePrometheusMetricsResponse.java
-rw----     0.0 fat     1220 tx      603 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/NodePrometheusRequestBuilder.java
-rw----     0.0 fat    10593 tx     2640 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/TransportNodePrometheusMetricsAction.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/admin/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/admin/indices/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/admin/indices/stats/
-rw----     0.0 fat     1213 tx      647 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/admin/indices/stats/PackageAccessHelper.java
-rw----     0.0 fat      692 tx      433 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/admin/indices/stats/package-info.java
-rw----     0.0 fat      718 tx      445 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/action/package-info.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/plugin/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/plugin/prometheus/
-rw----     0.0 fat     3057 tx     1055 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/plugin/prometheus/PrometheusExporterPlugin.java
-rw----     0.0 fat      707 tx      439 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/plugin/prometheus/package-info.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/rest/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/rest/prometheus/
-rw----     0.0 fat     4456 tx     1491 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/rest/prometheus/RestPrometheusMetricsAction.java
-rw----     0.0 fat      722 tx      448 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/java/org/opensearch/rest/prometheus/package-info.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/plugin-metadata/
-rw----     0.0 fat      843 tx      479 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/main/plugin-metadata/plugin-security.policy
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/org/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/org/opensearch/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/org/opensearch/rest/
-rw----     0.0 fat     1384 tx      664 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/org/opensearch/rest/PrometheusRestHandlerClientYamlTestSuiteIT.java
-rw----     0.0 fat      669 tx      416 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/java/org/opensearch/rest/package-info.java
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/api/
-rw----     0.0 fat      317 tx      181 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/api/prometheus.metrics.json
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/
drwx---     0.0 fat        0 bx        0 stor 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/
-rw----     0.0 fat      832 tx      399 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/10_basic.yml
-rw----     0.0 fat     1791 tx      894 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/20_00_metrics.yml
-rw----     0.0 fat     4686 tx      792 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/20_10_cluster_settings_metrics_disabled.yml
-rw----     0.0 fat     2258 tx      637 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/20_11_index_level_metrics_disabled.yml
-rw----     0.0 fat     3765 tx      798 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_00_index_level_info.yml
-rw----     0.0 fat     4317 tx      544 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_10_index_indexing.yml
-rw----     0.0 fat     2998 tx      476 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_11_index_get.yml
-rw----     0.0 fat     4240 tx      525 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_12_index_search.yml
-rw----     0.0 fat     4316 tx      549 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_13_index_merges.yml
-rw----     0.0 fat     1656 tx      423 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_14_index_refresh.yml
-rw----     0.0 fat     1274 tx      398 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_15_index_flush.yml
-rw----     0.0 fat     3140 tx      488 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_16_index_querycache.yml
-rw----     0.0 fat     1310 tx      414 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_17_index_fieldata.yml
-rw----     0.0 fat      923 tx      381 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_18_index_completion.yml
-rw----     0.0 fat     1359 tx      469 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_19_index_segments.yml
-rw----     0.0 fat     1614 tx      423 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_20_index_suggest.yml
-rw----     0.0 fat     2059 tx      447 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_21_index_requestcache.yml
-rw----     0.0 fat     1336 tx      435 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_22_index_recovery.yml
-rw----     0.0 fat     2088 tx      445 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_23_index_translog.yml
-rw----     0.0 fat     1604 tx      424 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/30_24_index_warmer.yml
-rw----     0.0 fat     5657 tx      655 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/40_10_cluster_settings_disk_threshold.yml
-rw----     0.0 fat     6738 tx     1167 defN 22-Jan-30 16:40 opensearch-prometheus-exporter-master/src/test/resources/rest-api-spec/test/resthandler/40_20_cluster_settings_disk_watermark_bytes.yml
83 files, 212891 bytes uncompressed, 50024 bytes compressed:  76.5%

I download ZIP directly from Github. I upload zip in my linux worker. (Worker is offline)

For info, on my worker, i have multi opensearch nodes ( I don’t know if it impacts the installation of the plugin) :
ls -ltrh /opt/opensearch/
drwxr-x— 3 opensearch opensearch 4.0K Jan 26 16:58 90
drwxr-x— 3 opensearch opensearch 4.0K Jan 26 16:59 01
drwxr-x— 3 opensearch opensearch 4.0K Jan 26 16:59 02
drwxr-x— 3 opensearch opensearch 4.0K Jan 26 16:59 03

For plugin install, i go in master directory to launch :
cd /opt/opensearch/90/current/
./bin/opensearch-plugin install -b file:///images/opensearch-prometheus-exporter-master.zip -v

I wonder where I’m wrong

Best Regards

Gnarly

Oh … you downloaded sources, you need distribution ZIP from: https://github.com/aparo/opensearch-prometheus-exporter/releases

Thanks for all :slight_smile: My Bad !

So, it seems good now :

-> Installing file:///home/osadmin/prometheus-exporter-1.2.3.zip
-> Downloading file:///home/osadmin/prometheus-exporter-1.2.3.zip
Retrieving zip from file:///home/osadmin/prometheus-exporter-1.2.3.zip
- Plugin information:
Name: prometheus-exporter
Description: Export OpenSearch metrics to Prometheus
Version: 1.2.3
OpenSearch Version: 1.2.3
Java Version: 1.8
Native Controller: false
Extended Plugins: []
 * Classname: org.opensearch.plugin.prometheus.PrometheusExporterPlugin
Folder name:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.reflect.ReflectPermission suppressAccessChecks
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.
-> Installed prometheus-exporter with folder name prometheus-exporter

If i want monitoring cluster, i need to deploy this plugin on all nodes ?

Thanks

Correct, please check out the docs [1], [2].

[1] https://github.com/aparo/opensearch-prometheus-exporter#configure-the-prometheus-target
[2] https://github.com/aparo/opensearch-prometheus-exporter#install

Thanks for all.
I can plug my Prometheus on Metrics API.

(l)

After usage, I find it heavy with that :frowning:

I have 72 nodes, installed on classic jvm on physical worker. So i need to install plugin on 72 nodes.

The Elasticsearch PLugin is more light and no need to install it on any nodes.
Furthermore, number of metrics is also big, 35k metrics by nodes :open_mouth:

(Also On Prometheus configuration, i need to declare 72 nodes…)

Thanks.

@GnarlyCapricorne by Elasticsearch PLugin you mean [1] or something else? Thank you.

[1] GitHub - prometheus-community/elasticsearch_exporter: Elasticsearch stats exporter for Prometheus

Yes Exactly.
The old exporter (for Elasticsearch CE) works very well because it goes up the metrics about all nodes.

… Unlike the new exporter for opensearch which requires installation on all nodes.

Also, new exporter is very resource intensive compared to the old exporter.

Just a feedback for all user.

PS : If anyone know how to modify an exporter or has already done so? Somes metrics are useless… :frowning:

Thanks

Gnarly

Hi Gnarly,

I would be really interested in learning more details.

The OpenSearch exporter works the same way as the original Elasticsearch exporter. Even for Elasticsearch exporter the recommendation was to install it on all nodes (because generally the assumption is that Prometheus is scraping metrics from all cluster nodes thus the plugin must be present on every node).

Can you please share more details about your OpenSearch cluster - Prometheus setup? How is Prometheus discovering individual OpenSearch nodes? Do you provide static list of nodes to scrape or do you rely on dynamic node discovery?

What makes you think the OpenSearch Prometheus exporter has degraded performance?

Which version of OpenSearch are you using BTW?

PS : If anyone know how to modify an exporter or has already done so? Somes metrics are useless…

Feel free to provide more details about the problem and I can instruct you.

Regards,
Lukáš

Hii,

The OpenSearch exporter works the same way as the original Elasticsearch exporter.

The elasticsearch exporter…
was installed like a java process on one physcal machine Opensearch.
I can configure the listening port.
I use es.all option to retrieve all cluster and node stats. (i not load indices, settings…).
With es.all this exporter generated 22 000 metrics AND i have all metrics for all nodes !
(If i install elasticsearch exporter on another node, we will have 22 000 * 2 metrics scraped.)
For information, we all options set, elasticsearch exporter generate 60 000 metrics by default.

Synthesis :
With just one install of this exporter, i have all metrics of cluster and nodes !
Ok it is a spof, but I just have to install it on another machine to have HA.
Volume of metrics (with full options) : 60 000 * 2 elasticsearch exporter (like 2 for HA)

The Opensearch exporter…
was installed like a plugin on Opensearch. Very easy to use.
I cannot configure the listening port (is not a problem because i can use push gateway…).
Options to limits metrics like indices, settings must be write in opensearch.yml ? Dynamic updated settings go to 404 error from GitHub - aparo/opensearch-prometheus-exporter: OpenSearch Prometheus Exporter (fork of https://github.com/vvanholl/elasticsearch-prometheus-exporter)
BUT if plugin is not deploy on all nodes, only metrics about the node where exporter was installed are scraped.
Plugin Opensearch exporter generated by default, 60 000 metrics by nodes. (I have 72 nodes…)
So 4 320 000 metrics scraped at each scrapping.

Synthesis :
For cluster informations it’s the same behavior.
For nodes stats, you must deploy plugin on all nodes to have full metrics like elasticsearch exporter. (…more consumer)

For info : Opensearch 1.2.3
My opensearch cluster is composed :
18 Linux physical machine with 300 GB RAM. (java process)
72 nodes (4 on each machine) : 3 masters, 6 coordinators (4 for dedicated writting) and 63 data nodes.
Network private. (we must use Push Gateway to export Metrics to Prometheus).

Prometheus is not direcly accessible by physical machine, we have to pass through a push gateway.
==> I don’t know how to use dynamic discovery node and i not view documentation about that. (And we don’t have acces to Prometheus with machines Opensearch)

With Opensearch or Elasticsearch Plugin; each nodes send metrics to Push Gateway.
Compared to the elasticsearch exporter, Opensearch exporter represents a much higher load because each node sends its own 60 000 metrics by default.

For me and for the moment, the easiest to use remains the elasticsearch plugin to scrape clusters and nodes metrics.

I may be wrong but looking at the load versus information reported, the elasticsearch plugin seems more profitable

I think it lacks an es.all on Opensearch Plugin like the Elasticsearch plugin does. That way, with opensearch plugin, I would just have to install the plugin on my coordinator nodes (4) and i have HA and all nodes metrics!