How to create opnesearch cluster?

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch version 2.15.X
OS - linux-22.04

not able to configure cluster in opensearch ?

require help here?

Thanks in advanced

Describe the issue:

Configuration:

Relevant Logs or Screenshots:

2

Hi @Ekta,

Would you mind elaborating a bit more?
Feel free to share your config files for review.

Thanks,
mj

3

Hi @Mantas

please find the config as below both node config same

======================== OpenSearch Configuration =========================

NOTE: OpenSearch comes with reasonable defaults for most settings.

Before you set out to tweak and tune the configuration, make sure you

understand what are you trying to accomplish and the consequences.

The primary way of configuring a node is via this file. This template lists

the most important settings you may want to configure for a production cluster.

Please consult the documentation for further information on configuration options:

https://www.opensearch.org

---------------------------------- Cluster -----------------------------------

Use a descriptive name for your cluster:

cluster.name: opensearch-cluster

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

node.name: TEST02

Add custom attributes to the node:

#node.attr.rack: r1
node.roles: [“cluster_manager”,“data”,“ingest”]
#node.roles: [cluster_manager]

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/lib/opensearch

Path to log files:

path.logs: /var/log/opensearch

----------------------------------- Memory -----------------------------------

Lock the memory on startup:

bootstrap.memory_lock: true

Make sure that the heap size is set to about half the memory available

on the system and that the owner of the process is allowed to use this

limit.

OpenSearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

Set the bind address to a specific IP (IPv4 or IPv6):

network.host: 0.0.0.0
#network.bind_host: [local, site]
network.bind_host: 10.x.x.x
#network.bind_host: [local, site]

Set a custom port for HTTP:

#http.port: 9200

For more information, consult the network module documentation.

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when this node is started:

The default list of hosts is [“127.0.0.1”, “[::1]”]

discovery.seed_hosts: [“TEST02”,“TEST04”]
#discovery.type: single-node

Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:

cluster.initial_master_nodes: [“TEST02”]
#cluster.initial_cluster_manager_nodes: [“TEST02”]
cluster.initial_master_nodes: [“TEST02”]
#cluster.initial_cluster_manager_nodes: [“TEST02”]

For more information, consult the discovery and cluster formation module documentation.

---------------------------------- Gateway -----------------------------------

Block initial recovery after a full cluster restart until N nodes are started:

gateway.recover_after_nodes: 2

For more information, consult the gateway module documentation.

---------------------------------- Various -----------------------------------

Require explicit names when deleting indices:

#action.destructive_requires_name: true

---------------------------------- Remote Store -----------------------------------

Controls whether cluster imposes index creation only with remote store enabled

cluster.remote_store.enabled: true

Repository to use for segment upload while enforcing remote store for an index

node.attr.remote_store.segment.repository: my-repo-1

Repository to use for translog upload while enforcing remote store for an index

node.attr.remote_store.translog.repository: my-repo-1

---------------------------------- Experimental Features -----------------------------------

Gates the visibility of the experimental segment replication features until they are production ready.

OpenSearch.experimental.feature.segment_replication_experimental.enabled: false

Gates the functionality of a new parameter to the snapshot restore API

that allows for creation of a new index type that searches a snapshot

directly in a remote repository without restoring all index data to disk

ahead of time.

ahead of time.

OpenSearch.experimental.feature.searchable_snapshot.enabled: false

Gates the functionality of enabling extensions to work with OpenSearch.

This feature enables applications to extend features of OpenSearch outside of

the core.

OpenSearch.experimental.feature.extensions.enabled: false

Gates the optimization of datetime formatters caching along with change in default datetime formatter

Once there is no observed impact on performance, this feature flag can be removed.

OpenSearch.experimental.optimization.datetime_formatter_caching.enabled: false

Gates the functionality of enabling Opensearch to use pluggable caches with respective store names via setting.

OpenSearch.experimental.feature.pluggable.caching.enabled: false

######## Start OpenSearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

Plugins.security.ssl.transport.pemcert_filepath: esnode.pem
Plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
Plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
Plugins.security.ssl.http.enabled: true
Plugins.security.ssl.http.pemcert_filepath: esnode.pem
Plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
Plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
Plugins.security.allow_unsafe_democertificates: true
Plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: [‘CN=kirk,OU=client,O=client,L=test,C=de’]
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent,.plugins-ml-config,.plugins-ml-connector,.plugins-ml-controller,.plugins-ml-model-group,.plugins-ml-model,.plugins-ml-task,.plugins-ml-conversation-meta,.plugins-ml-conversation-interactions,.plugins-ml-memory-meta,.plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,.opendistro-alerting-alert*,.opendistro-anomaly-results*,.opendistro-anomaly-detector*,.opendistro-anomaly-checkpoints,.opendistro-anomaly-detection-state,.opendistro-reports-,.opensearch-notifications-, .opensearch-notebooks,.opensearch-observability,.ql-datasources,.opendistro-asynchronous-search-response*,.replication-metadata-store,.opensearch-knn-models,.geospatial-ip2geo-data*,.plugins-flow-framework-config,.plugins-flow-framework-templates,.plugins-flow-framework-state]
node.max_local_storage_nodes: 2
plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/cert/demo.crt
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/cert/demo.key
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/cert/demo_ca.crt
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/cert/demo.crt
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/cert/demo.key
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/cert/demo_ca.crt
plugins.security.allow_default_init_securityindex: true
Plugins.security.disabled: true
Plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/cert/node1.pem
Plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/cert/node1-key.pem
Plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/cert/root-ca.pem

Plugins.security.ssl.transport.enforce_hostname_verification: false
Plugins.security.ssl.http.enabled: true
Plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/cert/node1.pem
Plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/cert/node1-key.pem
Plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/cert/root-ca.pem
######## End OpenSearch Security Demo Configuration ########
~

4

I’ve cleaned up your config file (see below). Give it a go and report errors back here:



# ======================== OpenSearch Configuration =========================

cluster.name: opensearch-cluster

node.name: TEST02
# node.roles: ["cluster_manager", "data", "ingest"]

path.data: /var/lib/opensearch
path.logs: /var/log/opensearch

bootstrap.memory_lock: true

network.host: 0.0.0.0
# network.bind_host: 10.x.x.x

discovery.seed_hosts: ["TEST02", "TEST04"]
cluster.initial_master_nodes: ["TEST02"]

# gateway.recover_after_nodes: 2

# ---------------------------------- Security ----------------------------------
plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/cert/demo.crt
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/cert/demo.key
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/cert/demo_ca.crt
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/cert/demo.crt
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/cert/demo.key
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/cert/demo_ca.crt

plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true

plugins.security.authcz.admin_dn: ["CN=kirk,OU=client,O=client,L=test,C=de"]
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector, .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task, .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta, .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config, .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*, .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*, .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources, .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models, .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates, .plugins-flow-framework-state]
node.max_local_storage_nodes: 2

best,
mj

5

@Mantas did same setting but get below error logs

Exception during establishing a SSL connection: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

6

Ok, few things to check:

How did you generate your certificates?
have you followed the below(?):

as per Configuring TLS certificates - OpenSearch Documentation, the certificates’ " Path to the certificate’s key file (PKCS #8), which must be under the config directory, specified using a relative path. Required."

That is not the config directory

Lastly, make sure the openserch user/group has enough permissions to read the certificates.

best,
mj