Opensearch-performance-analyzer.service - java.lang.IllegalArgumentException: cannot add context to list

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch: 2.12.0
OS: Ubuntu 20.0.4.6 LTS

Describe the issue:

A fresh installation of OpenSearch 2.12.0 for testing purposes developed interest in the Performance Analyzer feature that led to the discovery of this exception:

user@ip-10-1-1-2:~$ sudo systemctl status --full --no-pager opensearch-performance-analyzer
● opensearch-performance-analyzer.service - OpenSearch Performance Analyzer
     Loaded: loaded (/lib/systemd/system/opensearch-performance-analyzer.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2024-03-13 20:54:46 UTC; 9min ago
   Main PID: 1735 (java)
      Tasks: 17 (limit: 9252)
     Memory: 80.3M
     CGroup: /system.slice/opensearch-performance-analyzer.service
             └─1735 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Xms4m -Xmx64m -XX:+UseSerialGC -Dlog4j.configurationFile=/etc/opensearch/opensearch-performance-analyzer/log4j2.xml -Xms64M -Xmx64M -XX:+UseSerialGC -XX:CICompilerCount=1 -XX:-TieredCompilation -XX:InitialCodeCacheSize=4096 -XX:MaxRAM=400m -Dopensearch.path.home=/usr/share/opensearch -Dopensearch.path.conf=/etc/opensearch -Dopensearch.distribution.type=deb -cp /usr/share/opensearch/lib/*:/usr/share/opensearch/performance-analyzer-rca/lib/* org.opensearch.performanceanalyzer.PerformanceAnalyzerApp

Mar 13 20:54:46 ip-10-1-1-2 systemd[1]: Started OpenSearch Performance Analyzer.
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]: Exception in thread "main" java.lang.IllegalArgumentException: cannot add context to list
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at jdk.httpserver/sun.net.httpserver.ContextList.add(ContextList.java:37)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at jdk.httpserver/sun.net.httpserver.ServerImpl.createContext(ServerImpl.java:276)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at jdk.httpserver/sun.net.httpserver.HttpServerImpl.createContext(HttpServerImpl.java:74)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at jdk.httpserver/sun.net.httpserver.HttpServerImpl.createContext(HttpServerImpl.java:39)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.createClientServers(PerformanceAnalyzerApp.java:354)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.createClientServers(PerformanceAnalyzerApp.java:319)
Mar 13 20:54:48 ip-10-1-1-2 performance-analyzer-agent-cli[1735]:         at org.opensearch.performanceanalyzer.PerformanceAnalyzerApp.main(PerformanceAnalyzerApp.java:112)
user@ip-10-1-1-2:~$

Configuration:

  • 3 Nodes
  • node.roles: [cluster_manager,data,ingest]

Relevant Logs or Screenshots:

-rw-r--r-- 1 opensearch opensearch 0 Mar 12 13:20 PerformanceAnalyzer.log

$ tail performance_analyzer_agent_stats.log
------------------------------------------------------------------------
Program=PerformanceAnalyzerAgent
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Wed, 13 Mar 2024 21:11:43 UTC
Time=1710364303325 msecs
Timing=total-time:1.710364303325E12/1
Counters=
EOE
$

This issue is now documented in a bug report:

This issue actually prevents PerformanceAnalyzer from listening for any requests on port 9600, which makes it unusable. By looking at the sources it is an obvious issue, as the code tries to add the same handler 2 times so, the second attempt throws an exception which is not caught, so app is not working after that. The actual bug needs to be opened in : GitHub - opensearch-project/performance-analyzer-rca: The Performance Analyzer RCA is a framework that builds on the Performance Analyzer engine to support root cause analysis (RCA) of performance and reliability problems for OpenSearch instances.