OpenSearch server rolling upgrade from release 1.3.1 to release 2.10.0.
Describe the issue:
We are attempting to use a modified rolling upgrade process to upgrade from release 1.3.1 to release 2.10.0. The modified process we are using is to add three new nodes to the cluster running at 1.3.1 and at the new release of 2.10.0 for a total of 6 nodes. Then drop down one node at a time to get back to three nodes and ultimately running at 2.10.0. The reason for this approach is that we are using an aws nitro configuration that requires us to automatically delete our old volumes on shutdown in one of our aws environments. The problem we are having is that during the modified rolling upgrade process, there is an attempt move replica shards to an OpenSearch 1.3.1 instance (lower release) that ultimately fails. This appears to be much like the multi-release cluster issues described in other documentation I have found. Is there a way to prevent shard allocation attempts from higher releases to lower releases?
I have seen the option of using share segmentation, reindexing that index, and then using the parameter cluster.routing.allocation.shard_movement_strategy: REPLICA_FIRST.
We do not have the option of reindexing the index with the parameters listed above. Is there any way to prevent replica share allocation to the old release and simply copy the replica to the new release? even by hand? if needed?
Thanks in advance for reading this.
Configuration:
Relevant Logs or Screenshots: