Understanding shard allocation algorithm and disk usage

OpenSearch
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.8

Describe the issue:

I’d like to understand more about how OpenSearch decides where to allocate shards, and which index settings I can use to adjust the heuristics. Specifically, I’d like to balance the disk space available between my nodes more. I noticed that in ElasticSearch the cluster.routing.allocation.balance.disk_usage setting can be used to increase the weight of available disk space when determining where to allocate the shard. That setting is absent in OpenSearch. I’m wondering:

  1. Does the absence of this setting mean that disk usage is not a factor in the OpenSearch allocation algorithm, or is this just not configurable?
  2. Is there another way to adjust the allocation algorithm to take disk usage into account? Potentially by lowering cluster.routing.allocation.balance.shard?

Configuration:

Relevant Logs or Screenshots:

2

@ifrancke I dont think there are any options in opensearch that rebalances based on disk usage, apart from the disk.watermark.

Have you had a look at the following configuration options:

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.balance.index": 0.45,
    "cluster.routing.allocation.balance.shard": 0.55,
    "cluster.routing.allocation.balance.threshold": 1.0
  }
}

You can use the above configuration options spread the shards, but this is still not disk aware.
The is already a feature request for this.

Hope this helps

3

Thanks, that’s helpful. So, even without the configuration, is disk space a parameter in the allocation algorithm?