Need help 3 cluster node

OpenSearch
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Hi
I’m trying to setup a 3 nodes cluster all of them as cluster_manager , data . Somehow kibana shows only 1 node.
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles cluster_manager name
172.31.35.94 3 75 0 0.18 0.21 0.23 dm cluster_manager,data * node-1

this is my opensearch.yml for all 3 nodes
#cluster.initial_cluster_manager_nodes: [“node-1”, “node-2”]
cluster.initial_cluster_manager_nodes: [“node-1”,“node-2”,“node-3”]
#discovery.seed_hosts: [“172.31.35.94”, “172.31.44.145”, “172.31.34.238”]
discovery.seed_hosts: [“node-1”, “node-2”, “node-3”]

Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:

cluster.initial_cluster_manager_nodes: [“node-1”,“node-2”,“node-3”]
#discovery.seed_hosts: [“172.31.35.94”, “172.31.44.145”, “172.31.34.238”]
discovery.seed_hosts: [“node-1”, “node-2”, “node-3”]
#cluster.initial_cluster_manager_nodes: [“node-1”, “node-2”]

#discovery.seed_hosts: [“host1”, “host2”]
#discovery.type: single-node

Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:

cluster.initial_cluster_manager_nodes: [“node-1”,“node-2”,“node-3”]
#discovery.seed_hosts: [“172.31.35.94”, “172.31.44.145”, “172.31.34.238”]
discovery.seed_hosts: [“node-1”, “node-2”, “node-3”]

For more information, consult the discovery and cluster formation module documentation.

2

Hi @nkalvaku,

Could you share your full opensearch.yml (all three would be great) and your opensearch_dashboards.yml

when sharing please use the “Preformatted text (Ctrl + e)”:

image

Thanks,
mj

3 
root@opensearch-d1:~# grep -v "^#" /etc/opensearch/opensearch.yml
cluster.name: opensearch-cluster
node.name: opensearch-d1
node.roles: [data]
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
  #network.bind_host: "172.31.45.120"
discovery.seed_hosts: ["172.31.45.120","172.31.40.94","172.31.40.32"]
cluster.initial_cluster_manager_nodes: ["opensearch-cluster_manager"]

 WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state]
node.max_local_storage_nodes: 3
4 
root@opensearch-d2:~# grep -v "^#" /etc/opensearch/opensearch.yml
cluster.name: opensearch-cluster
node.name: opensearch-d2
node.roles: [data]
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
  #network.bind_host: "172.31.40.32"
discovery.seed_hosts: ["172.31.45.120","172.31.40.94","172.31.40.32"]
cluster.initial_cluster_manager_nodes: ["opensearch-cluster_manager"]

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state]
node.max_local_storage_nodes: 3
5 
root@opensearch-clustermanager:~# grep -v "^#" /etc/opensearch/opensearch.yml
cluster.name: opensearch-cluster
node.name: opensearch-cluster_manager
node.roles: [ cluster_manager, data  ]
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
discovery.seed_hosts: ["172.31.45.120","172.31.40.94","172.31.40.32"]
cluster.initial_cluster_manager_nodes: ["opensearch-cluster_manager"]


plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state]
node.max_local_storage_nodes: 3
6

Hi @Mantas
Please find details below.

7

These files are from a new 3 node cluster . Dashboard is not setup yet.

8

My intention is to configure 3 nodes cluster all being cluster_manager , data with dashboard. Question regarding dashboard , do we need to install it on any of 3 nodes , all of them or on a separate server ? I’m testing this on Ubuntu 22 on AWS. So far tried multiple attempts failing at some point. Don’t find a perfect doc anywhere .

9

@Mantas I’m good now . Thanks for your time.