Hey all, basicly we have daily indices which has 1 primary and 2 replicas, 3 node cluster. Each node gets 1 shard, what we noticed all primaries are concentrated on 1 node. For example node-2 has all primaries and node-0 and node-1 has replicas then node-2 restarts and we see all primaries promoted on node-0 but not even 1 promoted on node-1. Why is that anyone knows? when there are 2 candidate replicas how OpenSearch decides which one to promote?
1 Like
@bakili the distribution if primary shards is not enabled by default, therefore cluster-manager node decides which shards to promote based on the first active, in-sync replica copy
Have a look at configuration option cluster.routing.allocation.balance.prefer_primary, more details are available here
Hope this helps
@Anthony thank you
shards to promote based on the first active, in-sync replica copy
is it documented how exactly it decides it? generally is there any drawbacks of primaries concentrated on one node?
Have a look at configuration option
cluster.routing.allocation.balance.prefer_primary, more details are available here
this is safe? is it have any drawbacks? because we have indices/shards from different size groups, some index patterns has 30-40GB shards while some has just 20mb shards, enabling primary shard balancing can cause storage usage disbalance?
1 Like
@bakili it is generally recommended to let cluster-manager decide which shards to promote to primaries. The internal mechanism is designed to be efficient for storing and searching data. Therefore these options are set to false by default.
There is some more information on the Tracking in-sync shard copies here
and information on index and search workloads here that might be of use.
1 Like