Syncing two clusters

[szurokl]

Hello OpenSearch community,

I would like to ask what is the current recommended way to synchronize/backup indices between two OpenSearch clusters? (The more frequent the synchronization can be, the better + I would like to be able to automatically delete old data)

I know CCR exists, but I run into problems with it (auto deletion is not a part of the plugin yet, ISM policy unable to delete replicated index, due to it being readonly, CCR jobs get stuck in the background trying to replicate non-existent indices).

I considered using elasticdump, but it seems to be kind of slow, or configuring logstash with separate pipelines to write to multiple clusters.

If anyone has experience with similar use case, I would welcome some recommendations.

[Gsmitt]

Hey @szurokl

@szurokl

synchronize/backup indices between two OpenSearch clusters?

Since you stated two clusters, you could configure Data Sources shown here.

Basically connect to another opensearch/elasticsearch node. once completed create a index pattern, then pretty much create anything from that data on one node.

I havent tried to sync indices yet, but you could configure snapshot policies, Make sure you created a index pattern then create you snapshot policy from that. Not sure if it will work, I have not tried it yet.