Cluster not forming

danrk · October 16, 2025, 1:13am

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 3.2.0

Describe the issue: cluster not forming after following steps 1 thru 5, any idea why not?

Configuration: as it says in the documentation

Relevant Logs or Screenshots:

Anthony · October 16, 2025, 9:30am

@danrk please provide further details:

what method are you using to deploy? (helm, k8s, docker, rpm)
Where are you deploying this
Can you provide the logs generated when the cluster is attempting to start.
Can you provide copies of opensearch.yml (and opensearch_dashboards.yml if OSD is used)

danrk · October 16, 2025, 7:24pm

I am using the method from the official documentation.
Deploying this on 3 Linux VMs on Proxmox
here is
a sample

step 1: name a cluster

cluster.name: os-cluster-1

step 2: set nodes attributes

node.name: node1
node.roles: [ data, ingest, cluster_manager ]

step 3: bind cluster to specific IP address

network.bind_host: 192.168.1.41

step 4: configure discovery hosts and initial cluster manager

discovery.seed_hosts:
- 192.168.1.41
- 192.168.1.42
- 192.168.1.43
  cluster.initial_cluster_manager_nodes: [ “node2” ]
step 5: start the cluster

[root@node1 ~]# systemctl is-active opensearch
active

[root@node3 opensearch]# grep -E ‘node1|node2’ os-cluster-1.log
[2025-10-16T15:19:22,374][WARN ][o.o.d.HandshakingTransportAddressConnector] [node3] [connectToRemoteMasterNode[192.168.1.41:9300]] completed handshake with [{node1}{uBsrn9LNSf68D34EGeqvsg}{uDxiEm8uQSCGPtk_t4y4Dg}{127.0.0.1}{127.0.0.1:9300}{dim}{shard_indexing_pressure_enabled=true}] but followup connection failed
org.opensearch.transport.ConnectTransportException: [node1][127.0.0.1:9300] connect_exception
[2025-10-16T15:19:22,377][WARN ][o.o.d.HandshakingTransportAddressConnector] [node3] [connectToRemoteMasterNode[192.168.1.42:9300]] completed handshake with [{node2}{ayslqHlVQpe_X1hqEzsmmA}{kqQ5rUlBQJa76ZWG_hCVJA}{127.0.0.1}{127.0.0.1:9300}{dim}{shard_indexing_pressure_enabled=true}] but followup connection failed
org.opensearch.transport.ConnectTransportException: [node2][127.0.0.1:9300] connect_exception
[root@node3 opensearch]# date
Thu Oct 16 03:21:05 PM EDT 2025

Anthony · October 17, 2025, 8:35am

@danrk can you link the documentation you are using, as there are a number of official ways to deploy this. Can you also include the opensearch.yml file surrounded in code block tags.

danrk · October 17, 2025, 7:35pm

here is the guide: Creating a cluster - OpenSearch Documentation

danrk · October 18, 2025, 11:14am

This text will be hidden

``

network.bind_host: 192.168.1.41

cluster.name: os-cluster-1

node.name: node1

cluster.initial_cluster_manager_nodes: \[ “node2” \]

discovery.seed_hosts:

* 192.168.1.41
* 192.168.1.42
* 192.168.1.43
  node.roles: \[ data, ingest, cluster_manager \]

plugins.security.nodes_dn:

* ‘CN=node\*,OU=lab,C=US’

path.data: /var/lib/opensearch

path.logs: /var/log/opensearch

######## Start OpenSearch Security Demo Configuration ########

# WARNING: revise all the lines below before you go into production

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: \[all_access, security_rest_api_access\]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: \[.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
.plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
.plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
.plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
.opendistro-alerting-alert\*, .opendistro-anomaly-results\*, .opendistro-anomaly-detector\*,
.opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
.opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
.opendistro-asynchronous-search-response\*, .replication-metadata-store, .opensearch-knn-models,
.geospatial-ip2geo-data\*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
.plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache\]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

# custom certs

plugins.security.ssl.transport.pemcert_filepath: node1_crt.pem
plugins.security.ssl.transport.pemkey_filepath: node1_prv.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root_ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: node1_crt.pem
plugins.security.ssl.http.pemkey_filepath: node1_prv.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root_ca.pem
plugins.security.allow_unsafe_democertificates: false
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: \[‘CN=admin,OU=Lab,C=US’\]

Anthony · October 20, 2025, 11:38am

@danrk are you able to start a single node cluster? Do you have one node already running (and working) and trying to add another node to it? Are you using debian or rpm from the list here

lgrullonbb · November 7, 2025, 4:36am

I am using rpm and facing the same issues, single nodes works fine but when dealing with multinode I am not sure what I am missing, I try just about everything I can find in the docs and via google. Please assist.

I am running 3 nodes in AWS using RHEL 8.10 and latest OpenSearch with Dashboard.

Anthony · November 7, 2025, 10:37am

@lgrullonbb can you provide your opensearch.yml files for the 3 nodes? redact any sensitive details.

lgrullonbb · November 7, 2025, 9:59pm

See the config for the 3 nodes:

Node 1 Config

# ======================== OpenSearch Configuration =========================
#
# NOTE: OpenSearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.opensearch.org
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: bsi-prd-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node1
#
#
# Cluster-manager-eligible data nodes that will also be used for ingesting data
#
node.roles: [ cluster_manager, data, ingest ]
#
# Add custom attributes to the node:
#
#
network.bind_host: [ 10.123.80.97 ]
#
#
#discovery.seed_hosts: ["10.123.80.97", "10.123.80.143", "10.123.80.26"]
#
#
# discovery of hosts across the cluster
cluster.initial_cluster_manager_nodes: ["node1", "node2", "node3"]
#node.attr.rack: r1
#
action.auto_create_index: false
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/opensearch
#
# Path to log files:
#
path.logs: /var/log/opensearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
#
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_data_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the optimization of datetime formatters caching along with change in default datetime formatter
# Once there is no observed impact on performance, this feature flag can be removed.
#
#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: false
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
# Bind OpenSearch to the correct network interface. Use 0.0.0.0
# to include all available interfaces or specify an IP address
# assigned to a specific interface.
network.host: 0.0.0.0

# Unless you have already configured a cluster, you should set
# discovery.type to single-node, or the bootstrap checks will
# fail when you try to start the service.
#discovery.type: single-node

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: true

Node 2 config

# ======================== OpenSearch Configuration =========================
#
# NOTE: OpenSearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.opensearch.org
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: bsi-prd-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node2
#
#
# Cluster-manager-eligible data nodes that will also be used for ingesting data
#
node.roles: [ cluster_manager, data, ingest ]
#
#
network.bind_host: ["10.123.80.143"]
#
#
#discovery.seed_hosts: ["10.123.80.97", "10.123.80.143", "10.123.80.26"]
#
#
# discovery of hosts across the cluster
cluster.initial_cluster_manager_nodes: ["node1", "node2", "node3"]
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
action.auto_create_index: false
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/opensearch
#
# Path to log files:
#
path.logs: /var/log/opensearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
#
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_data_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the optimization of datetime formatters caching along with change in default datetime formatter
# Once there is no observed impact on performance, this feature flag can be removed.
#
#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: false
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
####### End OpenSearch Security Demo Configuration ########
# Bind OpenSearch to the correct network interface. Use 0.0.0.0
# to include all available interfaces or specify an IP address
# assigned to a specific interface.
network.host: 0.0.0.0

# Unless you have already configured a cluster, you should set
# discovery.type to single-node, or the bootstrap checks will
# fail when you try to start the service.
#discovery.type: single-node

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: true

Node 3 Config

# ======================== OpenSearch Configuration =========================
#
# NOTE: OpenSearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.opensearch.org
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: bsi-prd-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node3
#
# Add custom attributes to the node:
#
#
# Cluster-manager-eligible data nodes that will also be used for ingesting data
#
node.roles: [ master, data, ingest ]
#
#
network.bind_host: ["10.123.80.26"]
#
#
#
discovery.seed_hosts: ["10.123.80.97", "10.123.80.143", "10.123.80.26"]
#
#
# discovery of hosts across the cluster
cluster.initial_cluster_manager_nodes: ["node1", "node2", "node3"]
#
#
#node.attr.rack: r1
#
action.auto_create_index: false
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/opensearch
#
# Path to log files:
#
path.logs: /var/log/opensearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
#
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_data_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the optimization of datetime formatters caching along with change in default datetime formatter
# Once there is no observed impact on performance, this feature flag can be removed.
#
#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: false
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state, .plugins-search-relevance-experiment, .plugins-search-relevance-judgment-cache]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
####### End OpenSearch Security Demo Configuration ########
# Bind OpenSearch to the correct network interface. Use 0.0.0.0
# to include all available interfaces or specify an IP address
# assigned to a specific interface.
network.host: 0.0.0.0

# Unless you have already configured a cluster, you should set
# discovery.type to single-node, or the bootstrap checks will
# fail when you try to start the service.
#discovery.type: single-node

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: true

Anthony · November 10, 2025, 12:22pm

@lgrullonbb can you confirm how the security configuration is being managed in these clusters? Is the same CA used to sign all the certificates? Or are you using demo_installation of the security separately on each node? Are you able to extract the available logs with any errors listed?

lgrullonbb · November 10, 2025, 2:11pm

Hi Anthony, using the demo environment. I havent dive into the cert storm yet, trying to get this working with a plain vanilla installation.

Anthony · November 10, 2025, 3:44pm

@lgrullonbb You are just missing the below config in each node:

discovery.seed_hosts: ["192.168.1.155","192.168.1.156","192.168.1.157"] # replace with your IPs

lgrullonbb · November 10, 2025, 4:04pm

Hi @Anthony ,

I have adjusted the config and still only see a single node as per below output:

Every 2.0s: curl -XGET http://10.123.80.97:9200/_cat/nodes?v -u admin:v4H1FitYm6licpoPGUWw --insecure                                                    w1bsioscprdn02: Mon Nov 10 08:04:17 2025

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0 100   253  100   253    0     0  19461      0 --:--:-- --:--:-- --:--:-- 19461
ip           heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles                  cluster_manager name
10.123.80.97           25          98  -1    0.04    0.23     0.14 dim       cluster_manager,data,ingest *               node1

lgrullonbb · November 10, 2025, 5:47pm

I finally got the three nodes to show up in cluster mode, the issue was that /var didnt have adecuate space. After expanding spacing via LVM they all join correctly.

Topic		Replies	Views
Cluster in 2 host not working OpenSearch	1	946	June 21, 2023
About setting up high availability Elasticsearch cluster Open Source Elasticsearch and Kibana	24	3777	June 15, 2021
Why can't I create a cluster? OpenSearch discuss , troubleshoot , configure , install	10	95	July 1, 2025
3node opensearch cluster, setup is done, but it is not communicate with each other i am installing it on ubantu gcp vm, please help OpenSearch troubleshoot	10	1682	June 13, 2024
Cluster formation failure Open Source Elasticsearch and Kibana	1	591	February 7, 2023

Cluster not forming

step 2: set nodes attributes

step 3: bind cluster to specific IP address

step 4: configure discovery hosts and initial cluster manager

step 5: start the cluster

Related topics