# Opensearch primary shard allocation

**URL:** <https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739>\
**Category:** OpenSearch\
**Tags:** configure, index-management\
**Created:** [April 6, 2024, 8:33am UTC](https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739 "2024-04-06T08:33:29Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![coredump17](https://avatars.discourse-cdn.com/v4/letter/c/b77776/32.png) [@coredump17](https://forum.opensearch.org/u/coredump17)\
**Post date:** [April 6, 2024, 8:33am UTC](https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739/1 "2024-04-06T08:33:29Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):  
2.13

**Describe the issue** :  
I have 4x nodes with JBOD NVME disks. Each disk has an instance of opensearch on them (container - data node). I set a attribute value of hostname to prevent a shard from replicating to another disk on the same host. Note: i since found ‘cluster.routing.allocation.same\_shard.host’.

I have heavy indexing, light searching on this cluster. To reduce CPU, i enabled ‘SEGMENT’ replication which greatly helped (great they fixed the index template bug in 2.11 skipping this setting!). My next problem is, i see some hosts get more ‘Primary’ shards than others. This increases CPU on that host as the replica no longer needs to index the same documents. Do we have any way of allowing opensearch to weight primaries using an host attribute?

ie: 4 hosts, 6x disks per host = 24 OS Data nodes  
Index = 12 shard. Place 3 primaries on each host ‘attribute’ weight, then work out where the replicas should go?

I seen ‘cluster.routing.allocation.balance.prefer\_primary’ setting but as each disk instance is essentially a node, this isnt helping.

---

<div class="post-metadata">

**Author:** ![gaobinlong](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gaobinlong/32/5242_2.png) [@gaobinlong](https://forum.opensearch.org/u/gaobinlong)\
**Post date:** [April 9, 2024, 7:10am UTC](https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739/2 "2024-04-09T07:10:09Z")

</div>

I think you can try to increase the number of primary shards to 24, each node could have a equal number of primary shards and then the hosts are balanced. The best practice about sharding is that the number of primary shards is Nx greater than the number of nodes.

---

<div class="post-metadata">

**Author:** ![coredump17](https://avatars.discourse-cdn.com/v4/letter/c/b77776/32.png) [@coredump17](https://forum.opensearch.org/u/coredump17)\
**Post date:** [April 9, 2024, 8:33am UTC](https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739/3 "2024-04-09T08:33:45Z")

</div>

Wondering how the balancing algorithm works?

24x nodes.  
24x primary , 24 x replica

would the primary shards get balanced first and then the secondary? If so, your suggestion might work. If not, then i could still end up with more primaries on the same node. I was hoping for some way to influence the balance decision making a little.

---

<div class="post-metadata">

**Author:** ![gaobinlong](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gaobinlong/32/5242_2.png) [@gaobinlong](https://forum.opensearch.org/u/gaobinlong)\
**Post date:** [April 10, 2024, 8:59am UTC](https://forum.opensearch.org/t/opensearch-primary-shard-allocation/18739/4 "2024-04-10T08:59:05Z")

</div>

If the number of primary shards is Nx greater than the number of nodes, then there’s more chance that each node has equal primary shards, even though both the primary and replicas are considered in shard allocation.  
For your case, the index level setting `index.routing.allocation.total_shards_per_node` can be used to control number of shards that will be allocated to a single node, for example, 1. set number of replicas to `0`, 2. set `total_shard_per_node` to `1`, 3. after the shards movement finishes, set number of replicas back to `1`, now each node holds only one shard for the index which has 12 primary shards and 1 replicas.