Opensearch 3.1 new installation on single node - it is not starting up

ak_amkum19 · August 20, 2025, 10:14am

I have just one node, so what would be the next step for me. Should i run the ./securityadmin_demo.sh( i didnt get the file name in the tools).when i checked my opensearch status it is failing (looking for .pem certificates)

Anthony · August 20, 2025, 10:37am

@ak_amkum19 The script should have created demo certificates and updated opensearch.yml file. Can you check this file and see if this was added. Then check if the certificates were indeed generated. All of this should be in /etc/opensearch by default.

ak_amkum19 · August 20, 2025, 11:34am

yes, i am able to see the security configs in the openseach.yml file and there are root ca.pem , kirk-key,pem,kirk.pem.. all these files are there in the said location.

When i try to start the opensearch, it is throwing up anerror. i checked the log,

    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:872) ~[opensearch-3.1.0.jar:3.1.0]
    ... 15 more

\[2025-08-20T09:19:45,222\]\[ERROR\]\[o.o.b.OpenSearchUncaughtExceptionHandler\] \[node-1\] uncaught exception in thread \[main\]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class \[org.opensearch.security.OpenSearchSecurityPlugin\]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:172) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:159) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:110) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) \~\[opensearch-cli-3.1.0.jar:3.1.0\]
at org.opensearch.cli.Command.main(Command.java:101) \~\[opensearch-cli-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:125) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:91) \~\[opensearch-3.1.0.jar:3.1.0\]
Caused by: java.lang.IllegalStateException: failed to load plugin class \[org.opensearch.security.OpenSearchSecurityPlugin\]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:881) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:820) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:615) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.(PluginsService.java:229) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.node.Node.(Node.java:536) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.node.Node.(Node.java:464) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:249) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:249) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:411) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:168) \~\[opensearch-3.1.0.jar:3.1.0\]
… 6 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74) \~\[?:?\]
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) \~\[?:?\]
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) \~\[?:?\]
at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:872) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:820) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:615) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.plugins.PluginsService.(PluginsService.java:229) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.node.Node.(Node.java:536) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.node.Node.(Node.java:464) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap$5.(Bootstrap.java:249) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:249) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:411) \~\[opensearch-3.1.0.jar:3.1.0\]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:168) \~\[opensearch-3.1.0.jar:3.1.0\]
… 6 more
Caused by: org.opensearch.OpenSearchException: No SSL configuration found
at org.opensearch.security.ssl.SslSettingsManager.loadConfigurations(SslSettingsManager.java:128) \~\[?:?\]
at org.opensearch.security.ssl.SslSettingsManager.buildSslContexts(SslSettingsManager.java:96) \~\[?:?\]
at org.opensearch.security.ssl.SslSettingsManager.(SslSettingsManager.java:83) \~\[?:?\]
at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.(OpenSearchSecuritySSLPlugin.java:248) \~\[?:?\]
“my-application.log” 332L, 37239B

Anthony · August 20, 2025, 12:41pm

@ak_amkum19 can you provide a copy of your opensearch.yml, redact any sensitive details if any are present.

ak_amkum19 · August 20, 2025, 3:33pm

# 

# Gates the functionality of enabling extensions to work with OpenSearch.

# This feature enables applications to extend features of OpenSearch outside of

# the core.

# 

#opensearch.experimental.feature.extensions.enabled: false

# 

# 

# Gates the optimization of datetime formatters caching along with change in default datetime formatter

# Once there is no observed impact on performance, this feature flag can be removed.

# 

#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false

######## Start OpenSearch Security Demo Configuration ########

# WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.authcz.admin_dn: \[‘CN=kirk,OU=client,O=client,L=test,C=de’\]
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: \[all_access, security_rest_api_access\]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: \[.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
.plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
.plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
.plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
.opendistro-alerting-alert\*, .opendistro-anomaly-results\*, .opendistro-anomaly-detector\*,
.opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
.opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
.opendistro-asynchronous-search-response\*, .replication-metadata-store, .opensearch-knn-models,
.geospatial-ip2geo-data\*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
.plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache\]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
“opensearch.yml” 154L, 6463B

ak_amkum19 · August 20, 2025, 5:19pm

plz find the opensearch.yml file


# ======================== OpenSearch Configuration =========================

# 

# NOTE: OpenSearch comes with reasonable defaults for most settings.

# Before you set out to tweak and tune the configuration, make sure you

# understand what are you trying to accomplish and the consequences.

# 

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

# 

# Please consult the documentation for further information on configuration options:

# 

https://www.opensearch.org

# 

# ---------------------------------- Cluster -----------------------------------

# 

# Use a descriptive name for your cluster:

# 

cluster.name: my-application

# 

# ------------------------------------ Node ------------------------------------

# 

# Use a descriptive name for the node:

# 

node.name: node-1

# 

# Add custom attributes to the node:

# 

#node.attr.rack: r1

# 

# ----------------------------------- Paths ------------------------------------

# 

# Path to directory where to store the data (separate multiple locations by comma):

# 

path.data: /var/lib/opensearch

# 

# Path to log files:

# 

path.logs: /var/log/opensearch

# 

# ----------------------------------- Memory -----------------------------------

# 

# Lock the memory on startup:

# 

bootstrap.memory_lock: true\\

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

# 

# OpenSearch performs poorly when the system is swapping the memory.

# 

# ---------------------------------- Network -----------------------------------

# 

# Set the bind address to a specific IP (IPv4 or IPv6):

# 

network.host: 0.0.0.0

# 

# Set a custom port for HTTP:

# 

http.port: 9200

# 

# For more information, consult the network module documentation.

# 

# --------------------------------- Discovery ----------------------------------

# 

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is \[“127.0.0.1”, “\[::1\]”\]

# 

discovery.seed_hosts: \[“x.x.x.x”\]

# 

# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:

# 

cluster.initial_cluster_manager_nodes: \[“node-1”\]

# 

# For more information, consult the discovery and cluster formation module documentation.

# 

# ---------------------------------- Gateway -----------------------------------

# 

######## Start OpenSearch Security Demo Configuration ########

# WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.authcz.admin_dn: \[‘CN=kirk,OU=client,O=client,L=test,C=de’\]
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: \[all_access, security_rest_api_access\]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: \[.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
.plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
.plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
.plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
.opendistro-alerting-alert\*, .opendistro-anomaly-results\*, .opendistro-anomaly-detector\*,
.opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
.opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
.opendistro-asynchronous-search-response\*, .replication-metadata-store, .opensearch-knn-models,
.geospatial-ip2geo-data\*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
.plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache\]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
72,1          30%

Anthony · August 20, 2025, 6:37pm

@ak_amkum19 please surround your configuration in code brackets.

It seems like opensearch is either:

Not picking up the correct opensearch.yml file, examine the output of the command sudo systemctl cat opensearch and see if there is any mention of OPENSEARCH_PATH_CONF.
Also you can execute the below to see what the configuration is for this:

sudo grep -R "OPENSEARCH_PATH_CONF" -n /etc/sysconfig /etc/default /usr/lib/systemd/system/opensearch.service /etc/systemd/system/opensearch.service.d || true

Use absolute path for certificates (although I would expect a different errors if that was the issue).

ak_amkum19 · August 22, 2025, 5:55am

@Anthony changing the path of the certs to absolute path.. made me progress . it started the Opensearch, and in the status it shows opensearch is running. `but when i do a curl command to test , using this command curl -X GET http://localhost:9200 -u ‘admin:xxxxxx!’ --insecure it says Opensearch security not initialized

Anthony · August 22, 2025, 9:29am

@ak_amkum19 I think the “http” in the above is a typo, and its “https” as the security is enabled.

If the security is not initialized, you should be able to do it manually using the steps here

If you get any errors, please paste them here in code snippet.

ak_amkum19 · August 23, 2025, 1:19am

@Anthony i am trying to run the command to initialize the SECURITY . I have the Demo certs in the Linux machine(etc/opensearch) location.

OPENSEARCH_JAVA_HOME=/usr/share/opensearch/jdk ./securityadmin.sh -cd /etc/opensearch/opensearch-security/ -cacert /etc/opensearch/root-ca.pem -cert /etc/opensearch/esnode.pem -key /etc/opensearch/esnode-key.pem -icl -nhnv

When i run this command , i get this error message, please let me know which certs i need to use to run this command

Error Message : Will connect to localhost:9200 … done
Connected as “CN=node-0.example.com,OU=node,O=node,L=test,C=de”
ERR: “CN=node-0.example.com,OU=node,O=node,L=test,C=de” is not an admin user
Seems you use a client certificate but this one is not registered as admin_dn
Make sure opensearch.yml on all nodes contains:
plugins.security.authcz.admin_dn:

““CN=node-0.example.com,OU=node,O=node,L=test,C=de””

ak_amkum19 · August 23, 2025, 4:31am

@Anthony I was able to successfully initialize the Security.

[root@xxxxxxx tools]# OPENSEARCH_JAVA_HOME=/usr/share/opensearch/jdk ./securityadmin.sh -cd /etc/opensearch/opensearch-security/ -cacert /etc/opensearch/root-ca.pem -cert /etc/opensearch/kirk.pem -key /etc/opensearch/kirk-key.pem -icl -nhnv
Security Admin v7
Will connect to localhost:9200 … done
Connected as “CN=kirk,OU=client,O=client,L=test,C=de”
OpenSearch Version: 3.1.0
Contacting opensearch cluster ‘opensearch’ and wait for YELLOW clusterstate …
Clustername: New-Application
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index does not exists, attempt to create it … done (0-all replicas)
Populate config from /etc/opensearch/opensearch-security/
Will update ‘/config’ with /etc/opensearch/opensearch-security/config.yml
SUCC: Configuration for ‘config’ created or updated
Will update ‘/roles’ with /etc/opensearch/opensearch-security/roles.yml
SUCC: Configuration for ‘roles’ created or updated
Will update ‘/rolesmapping’ with /etc/opensearch/opensearch-security/roles_mapping.yml
SUCC: Configuration for ‘rolesmapping’ created or updated
Will update ‘/internalusers’ with /etc/opensearch/opensearch-security/internal_users.yml
SUCC: Configuration for ‘internalusers’ created or updated
Will update ‘/actiongroups’ with /etc/opensearch/opensearch-security/action_groups.yml
SUCC: Configuration for ‘actiongroups’ created or updated
Will update ‘/tenants’ with /etc/opensearch/opensearch-security/tenants.yml
SUCC: Configuration for ‘tenants’ created or updated
Will update ‘/nodesdn’ with /etc/opensearch/opensearch-security/nodes_dn.yml
SUCC: Configuration for ‘nodesdn’ created or updated
Will update ‘/audit’ with /etc/opensearch/opensearch-security/audit.yml
SUCC: Configuration for ‘audit’ created or updated
Will update ‘/allowlist’ with /etc/opensearch/opensearch-security/allowlist.yml
SUCC: Configuration for ‘allowlist’ created or updated
SUCC: Expected 9 config types for node {“updated_config_types”:[“allowlist”,“tenants”,“rolesmapping”,“nodesdn”,“audit”,“roles”,“actiongroups”,“config”,“internalusers”],“updated_config_size”:9,“message”:null} is 9 ([“allowlist”,“tenants”,“rolesmapping”,“nodesdn”,“audit”,“roles”,“actiongroups”,“config”,“internalusers”]) due to: null
Done with success

When i try connecting the opensearch url through curl command(in the local host only) i am getting the Unauthorized access message

ak_amkum19 · August 25, 2025, 3:44am

@Anthony -Thanks for all the help. I was able to finally get the output for my curl command , running it in local host. The issue with Unauthorized access was the admin password. I changed the password using the hash command and added up in the internal_users.yml file. Restarted the openseach, ran the .securityadmin.sh and it worked

Thanks once again for all the help

ak_amkum19 · August 25, 2025, 7:06am

@Anthony , in my opensearch.yml file, in the network section “ network.host : 0.0.0.0 , do i need to add in my local host address here. currently it is commented out. should i add my local host ip address, so i can access the opensearch cluster remotely from a windows machine

ak_amkum19 · August 25, 2025, 7:12am

whenever i uncomment the network host line, openseach doesnt start up

Anthony · August 25, 2025, 9:56am

@ak_amkum19 you should leave the above config as it it, this will enable you to connect to this cluster from outside the machine.

Topic		Replies	Views
Not able to start opensearch after configure tls and user Security configure , install	24	4373	November 11, 2022
While install opensearch2.12.0 its shows fail to connect the port 9200 Security install	34	2018	April 2, 2025
OpenSearch Cluster Security not initialized Security install , index-management	7	5433	August 14, 2024
Opensearch in get error failed to load plugin class OpenSearch	3	902	November 22, 2024
Not able to login with default password in OpenSearch Dashboard2.3 Security	21	1591	November 2, 2022

Opensearch 3.1 new installation on single node - it is not starting up

Related topics