Hey,
We have been spinning up a new opensearch cluster that is supposed to replace the old one. We did a change a while ago to our ingesting pipeline so that both opensearch clusters are receiving new logs.
The question now is how we can transfer all of the logs from the old cluster to the new one? The data is over 100TB so placing it in a snapshot repo (i.e. S3) is not really feasible. Is it possible to send the logs to the new cluster from the old?
Hey @victor ,
There is a few ways in which this can be done, you can review your choices here - Migrate or upgrade - OpenSearch Documentation .
Giving the large dataset and not wanting to use snapshots, possible options to consider are using Migration Assistant - OpenSearch Documentation or Reindex document - OpenSearch Documentation . Both work well with large volumes of data and next to zero downtime.
But I would suggest you first review the docs and see which works best for you.
Leeroy.