Opensearch startup errors that disappear over time

i found this index using this command:

GET _cat/indices?expand_wildcards=open,hidden

But specified log line i didn’t see @pablo

@Ruslan1 Thanks for sharing all the info. I’ve tested your docker compose file on Debian 13 with 16 CPU cores, 24 GB of RAM and a Java Heap size of 10GB.
I had no errors in my logs that were reported in this case.

The only difference I see is the environment. I’m running my VM on ESXi 8 using a directly attached SSD drive. All my files are under the root filesystem.

According to your df command output, your /data folder is mapped as a separate virtual disk.
Is it a slower kind of disk?

Also, I’ve noticed that you’re using the opensearch image without the opensearchproject repo. Does it mean that you’re using your own Docker image registry? If yes, are these images exactly the same as the originals, or have you modified them?

Thanks for the reply. I have HDD disks on the node. I’ll replace them with an SSD and give you an answer.

@pablo
I was able to test it on an SSD. Got the same result

{"log":"[2026-08-18T14:57:43,770][ERROR][o.o.i.i.ManagedIndexCoordinator] [2-opensearch01] Failed to get ISM policies with templates: Failed to execute phase [query], all shards failed\n","stream":"stdout","time":"2026-08-18T14:57:43.770143394Z"}

And i get:

[2026-08-18T14:57:33,344][ERROR][o.o.p.c.c.PluginSettings ] [2-opensearch01] Loading config file /usr/share/opensearch/config/opensearch-performance-analyzer/performance-analyzer.properties failed with error: org.opensearch.performanceanalyzer.commons.config.ConfigFileException: java.io.FileNotFoundException: /usr/share/opensearch/config/opensearch-performance-analyzer/performance-analyzer.properties (No such file or directory). Disabling plugin.
[2026-08-18T14:57:33,344][INFO ][o.o.p.c.c.PluginSettings ] [2-opensearch01] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: false, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
[2026-08-18T14:57:33,556][ERROR][o.o.p.c.c.StatsCollector ] [2-opensearch01] Error in loading metadata for folderLocation: /usr/share/opensearch/config/opensearch-performance-analyzer/, fileLocation: plugin-stats-metadata
java.io.FileNotFoundException: /usr/share/opensearch/config/opensearch-performance-analyzer/plugin-stats-metadata (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:185)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:139)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:109)
        at org.opensearch.performanceanalyzer.commons.collectors.StatsCollector.loadMetadata(StatsCollector.java:158)
        at org.opensearch.performanceanalyzer.commons.collectors.StatsCollector.instance(StatsCollector.java:73)
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerPlugin.scheduleRcaCollectors(PerformanceAnalyzerPlugin.java:268)
        at org.opensearch.performanceanalyzer.PerformanceAnalyzerPlugin.<init>(PerformanceAnalyzerPlugin.java:214)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
        at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:919)
        at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:870)
        at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:653)
        at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:227)
        at org.opensearch.node.Node.<init>(Node.java:567)
        at org.opensearch.node.Node.<init>(Node.java:495)
        at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:252)
        at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:252)
        at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:414)
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:168)
        at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:159)
        at org.opensearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:110)
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
        at org.opensearch.cli.Command.main(Command.java:101)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:125)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:91)