Keystore.tmp issue

Security
1

Hi community!

I am running into an issue when deploying elastic search with security / alerting plugin on Openshift. I believe my configuration is setup correctly (config.yml / elasticsearch.yml) and mounted.

The error I am receiving

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.FileSystemException: /usr/share/elasticsearch/config/elasticsearch.keystore.tmp: Read-only file system
Likely root cause: java.nio.file.FileSystemException: /usr/share/elasticsearch/config/elasticsearch.keystore.tmp: Read-only file system
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
	at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
	at java.base/java.nio.file.Files.newOutputStream(Files.java:222)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:411)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:407)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:255)
	at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:462)
	at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:232)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:289)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)

I spent the better half of my day trying to figure this out. Any help would be greatly appreciated .

2

Hello @williejay2009

Have you found a solution for this issue?

Read-only filesystem could mean that backing filesystem got corrupted due to unexpected restart.
Have you run a file check?

3

Hello, today updating image from docker opensearch 1.2.0 to 1.2.1 inside a data node i have the same error that i post here

Disabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
Enabling OpenSearch Security Plugin
sed: cannot rename /usr/share/opensearch/config/sedvIxpiT: Device or resource busy
Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.FileAlreadyExistsException: /usr/share/opensearch/config/opensearch.keystore.tmp
Likely root cause: java.nio.file.FileAlreadyExistsException: /usr/share/opensearch/config/opensearch.keystore.tmp
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:94)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
        at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
        at java.base/java.nio.file.Files.newOutputStream(Files.java:224)
        at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
        at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
        at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
        at org.opensearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:524)
        at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:276)
        at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
        at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
        at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
        at org.opensearch.cli.Command.main(Command.java:101)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Killing performance analyzer process 12
OpenSearch exited with code 1
Performance analyzer exited with code 14
4

@ynuyasha This error doesn’t look the same. Would you mind opening a new thread and posting the error there?
Also, please share your docker-compose file.

5

@pablo done thanks Keystore.tmp error upgrading 1.2.0 to 1.2.1