Hello,
some time ago Elastic released formal model for Elasticsearch: https://github.com/elastic/elasticsearch-formal-models. I am not sure if it is still maintained (seems the last update was almost 2 years ago). However, is there any plan to build similar model for OpenSearch? The question is how changes to distributed consensus and replication code will be evaluated and verified going forward.
Regards,
Lukáš
[I’m going to move this to the opensearch category - I think it’ll get more visibility.]
No work that I know of on that front. From the perspective of data replication and cluster coordination I don’t believe there is any plans to change anything away from what Elasticsearch did at least for the 1.x versions. Where it might depart is the implementation of CCR (not yet a part of OpenSearch but on the roadmap).
Is that a requirement or something you or something you’re interested in working on?
Interesting- I had never looked in that repo before.
I know we have had some ideas thrown around to provide a better shard allocation and distribution algorithm that would use the performance analyzer metrics (resource usage by shard and node load) to better distribute shards to help avoid creating hot spots and skew which are common if you are not careful today. I am not sure if we had anyone raise their hand say they are going to work on it yet. I am sure there are dozens of optimizations like that to be had. Is this an area you are interested in Lukáš? - would be curious if you had some other ideas.
Carl