hi, tell me please. what roles and parameters should be specified for failover cluster of 6 nodes? I am interested in the cluster manager role, in the event of a failure of one manager, I need the cluster to be managed by another with the same role. but at the moment my cluster is down after losing one node manager.
the cluster consists of 6 nodes. 1 coordinator + ingest, 2 manager + data, 3 data
@maxim Could you share the output of API _cat/nodes?
Please have a look at the OpenSearch documentation and described best practices.
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles cluster_manager name
71 96 1 0.07 0.04 0.00 d data - os-data-node3
8 96 1 0.16 0.08 0.04 dm cluster_manager,data - os-manager-data-node2
17 99 1 0.00 0.00 0.00 d data - os-data-node4
13 99 1 0.01 0.03 0.00 d data - os-data-node5
54 98 3 0.03 0.03 0.00 i coordinating,ingest - os-coordinator-node
19 99 1 0.53 0.45 0.20 dm cluster_manager,data * os-manager-data-node1
@maxim You have 2 master nodes. If you loose any of the nodes the cluster will get RED.
Did you have a look at the best practices?
Three dedicated cluster manager nodes in three different zones is the right approach for almost all production use cases. This configuration ensures your cluster never loses quorum. Two nodes will be idle for most of the time except when one node goes down or needs some maintenance.
@pablo yes, I read it, but as I understood what I read after the loss of one master, with this configuration, the role of the master should be taken over by the second server with the same role
but it doesn’t happen and it’s not clear to me what to do, set up zoning and assign a master to each zone, and the coordinator will act as a connecting node and a distributing node?
@maxim As per the documentation, your minimum number of master nodes should be 3 nodes to support failover. With 2 master nodes, the cluster will become RED once one of the nodes will fail.
I think the documentation by zoning refers to the physical placement of the nodes.