Unable to start opensearch from command line (Error: plugins.security.ssl.transport.keystore_filepath)

OpenSearch
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

opensearch-2.8.0

Describe the issue:
During the startup of opensearch, I am getting the following exception. Any help is much appreciated.

org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:184) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.8.0.jar:2.8.0]
at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103) ~[opensearch-2.8.0.jar:2.8.0]
Caused by: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:791) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.(PluginsService.java:195) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:454) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:381) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.8.0.jar:2.8.0]
… 6 more
… 6 more
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.(PluginsService.java:195) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:454) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:381) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.8.0.jar:2.8.0]
… 6 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:420) ~[?:?]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:256) ~[?:?]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.(DefaultSecurityKeyStore.java:177) ~[?:?]
at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
at org.opensearch.security.OpenSearchSecurityPlugin.(OpenSearchSecurityPlugin.java:266) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.(PluginsService.java:195) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:454) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:381) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.8.0.jar:2.8.0]
… 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:420)
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:256)
at org.opensearch.security.ssl.DefaultSecurityKeyStore.(DefaultSecurityKeyStore.java:177)
at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.(OpenSearchSecuritySSLPlugin.java:218)
at org.opensearch.security.OpenSearchSecurityPlugin.(OpenSearchSecurityPlugin.java:266)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782)
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731)
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533)
at org.opensearch.plugins.PluginsService.(PluginsService.java:195)
at org.opensearch.node.Node.(Node.java:454)
at org.opensearch.node.Node.(Node.java:381)
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
at org.opensearch.cli.Command.main(Command.java:101)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /home/acayci/opensearch/opensearch-2.8.0/logs/opensearch.log
[acayci@acayci-dev bin]$

Configuration:

here are the lines from opensearch.yml file

plugins.ml_commons.model_access_control_enabled: true
plugins.ml_commons.allow_registering_model_via_url: true
plugins.ml_commons.allow_registering_model_via_local_file: true

Relevant Logs or Screenshots:

2

Rest of the exception

... 6 more

Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.(PluginsService.java:195) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:454) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:381) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.8.0.jar:2.8.0]
… 6 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:420) ~[?:?]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:256) ~[?:?]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.(DefaultSecurityKeyStore.java:177) ~[?:?]
at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
at org.opensearch.security.OpenSearchSecurityPlugin.(OpenSearchSecurityPlugin.java:266) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.plugins.PluginsService.(PluginsService.java:195) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:454) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.node.Node.(Node.java:381) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.8.0.jar:2.8.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.8.0.jar:2.8.0]
… 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:420)
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:256)
at org.opensearch.security.ssl.DefaultSecurityKeyStore.(DefaultSecurityKeyStore.java:177)
at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.(OpenSearchSecuritySSLPlugin.java:218)
at org.opensearch.security.OpenSearchSecurityPlugin.(OpenSearchSecurityPlugin.java:266)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:782)
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:731)
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:533)
at org.opensearch.plugins.PluginsService.(PluginsService.java:195)
at org.opensearch.node.Node.(Node.java:454)
at org.opensearch.node.Node.(Node.java:381)
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
at org.opensearch.cli.Command.main(Command.java:101)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /home/acayci/opensearch/opensearch-2.8.0/logs/opensearch.log
[acayci@acayci-dev bin]$

3

Hi @acayci12324,

Could you please share your opensearch.yml file?

Thanks,
mj

4

Hello MJ, I believe I knew the issue. Since I was using the tarball distribution, I wasn’t using ./opensearch-tar-install.sh to start the open search and therefore the opensearch.yml file did not have the demo certifications. The file has now all those details. Am I correct with my assessment?

# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem

Mustafa

1 Like
5

@acayci12324, looks correct.

Best,
mj

1 Like