[Solved] Performance Analyzer has no metrics (docker container)

Plugins Performance Analyzer
1

This is a client node running on top of k8s. I’m trying to get metrics from PA but there’s nothing from PA.
Version. OpenSearch 1.3.0

[opensearch@opensearch-cluster-client-0 ~]$ ls /dev/shm/performanceanalyzer/ -alh
total 0
drwxrwxr-x 2 opensearch opensearch 40 Apr 19 08:01 .
drwxrwxrwt 3 root       root       60 Apr 19 08:01 ..

[opensearch@opensearch-cluster-client-0 ~]$ export OPENSEARCH_HOME=./
[opensearch@opensearch-cluster-client-0 ~]$ ./plugins/opensearch-performance-analyzer/pa_bin/performance-analyzer-agent 
Using JAVA_HOME: /usr/share/opensearch/jdk
07:44:16.522 [main] ERROR org.opensearch.performanceanalyzer.PerformanceAnalyzerWebServer - Could not create HttpServer on port 9600
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method) ~[?:?]
        at sun.nio.ch.Net.bind(Net.java:459) ~[?:?]
        at sun.nio.ch.Net.bind(Net.java:448) ~[?:?]
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227) ~[?:?]
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80) ~[?:?]
        at sun.net.httpserver.ServerImpl.<init>(ServerImpl.java:101) ~[jdk.httpserver:?]
        at sun.net.httpserver.HttpServerImpl.<init>(HttpServerImpl.java:50) ~[jdk.httpserver:?]
        at sun.net.httpserver.DefaultHttpServerProvider.createHttpServer(DefaultHttpServerProvider.java:35) ~[jdk.httpserver:?]
        at com.sun.net.httpserver.HttpServer.create(HttpServer.java:137) ~[jdk.httpserver:?]
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerWebServer.createHttpServer(PerformanceAnalyzerWebServer.java:137) ~[performance-analyzer-rca-1.3.1.0.jar:?]
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerWebServer.createInternalServer(PerformanceAnalyzerWebServer.java:46) [performance-analyzer-rca-1.3.1.0.jar:?]
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.createClientServers(PerformanceAnalyzerApp.java:322) [performance-analyzer-rca-1.3.1.0.jar:?]
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.createClientServers(PerformanceAnalyzerApp.java:294) [performance-analyzer-rca-1.3.1.0.jar:?]
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.main(PerformanceAnalyzerApp.java:129) [performance-analyzer-rca-1.3.1.0.jar:?]




[opensearch@opensearch-cluster-client-0 ~]$ cat plugins/opensearch-performance-analyzer/pa_config/performance-analyzer.properties 
# ======================== OpenSearch performance analyzer plugin config =========================

# NOTE: this is an example for Linux. Please modify the config accordingly if you are using it under other OS.

# Metrics data location
metrics-location = /dev/shm/performanceanalyzer/

# Metrics deletion interval (minutes) for metrics data.
# Interval should be between 1 to 60.
metrics-deletion-interval = 1

# If set to true, the system cleans up the files behind it. So at any point, we should expect only 2
# metrics-db-file-prefix-path files. If set to false, no files are cleaned up. This can be useful, if you are archiving
# the files and wouldn't like for them to be cleaned up.
cleanup-metrics-db-files = true

# WebService exposed by App's port
webservice-listener-port = 9600

# Port for RPC Communication
rpc-port = 9650

# Metric DB File Prefix Path location
metrics-db-file-prefix-path = /tmp/metricsdb_

https-enabled = false

# Setup the correct path for server certificates
certificate-file-path = none
private-key-file-path = none
#trusted-cas-file-path = none

# Setup the correct path for client certificates (by default, the client will just use the server certificates)
#client-certificate-file-path = specify_path
#client-private-key-file-path = specify_path
#client-trusted-cas-file-path = specify_path

# WebService bind host; default only to local interface
webservice-bind-host = 0.0.0.0

# Plugin Stats Metadata file name, expected to be in the same location
plugin-stats-metadata = plugin-stats-metadata

# Agent Stats Metadata file name, expected to be in the same location
agent-stats-metadata = agent-stats-metadata
2

Enabled by these API calls.

curl -XPOST https://localhost:9200/_plugins/_performanceanalyzer/cluster/config -H 'Content-Type: application/json' -d '{"enabled": true}' -u 'admin:admin' -k
curl -XPOST https://localhost:9200/_plugins/_performanceanalyzer/rca/cluster/config -H 'Content-Type: application/json' -d '{"enabled": true}' -u 'admin:admin' -k

curl -k -u admin:admin -XPOST https://localhost:9200/_plugins/_performanceanalyzer/batch/config -H 'Content-Type: application/json' -d '{"enabled": true}'
curl -k -u admin:admin -XPOST https://localhost:9200/_plugins/_performanceanalyzer/batch/cluster/config -H 'Content-Type: application/json' -d '{"enabled": true}'
1 Like