Should Dashboards and Data Prepper config point to all nodes of OpenSearch cluster?

OpenSearch
1

Versions: OpenSearch cluster and Dashboards 3.2, Data Prepper 2.12

Describe the issue: I currently have a simple OpenSearch test cluster with three nodes that have node types. I intend to make the cluster more robust by adding three dedicated manager nodes and take manager type away from the old nodes.

But I am unsure if I should point Data Prepper and Dashboards to all nodes or just some of them after changes.

Data Prepper has pipelines.yml sink like this:

sink:
  - opensearch:
      hosts:
        - https://192.168.114.101:9200
        - https://192.168.114.102:9200
        - https://192.168.114.103:9200

Should I add manager nodes here too even tough they don’t have ingest type?

And Dashboards has cluster nodes defined like this in opensearch_dashboards.yml:

opensearch.hosts: ["https://192.168.114.101:9200", "https://192.168.114.102:9200", "https://192.168.114.103:9200"]

Should I put all cluster nodes here too?

Configuration: Data Prepper sending logs to OpenSearch cluster.

2

@Strategios Thank you for the question. All client traffic should be routed to coordinating only nodes if these are present. If not, data nodes is the next option. The traffic should not be routed directly to the cluster-manager nodes. If multiple endpoints are specified (like your example), round robin approach will be used for load balancing.

Hope this answers your question

3

@Anthony Thanks for your response, that made things clear to me.

1 Like