I am new to the AWS OpenSearch and trying to achieve cross cluster replication between 2 domains in different AWS accounts.
As part of this, we are trying to enable the logstash input plugin to send out the domain index events to another account opensearch domain as provided on this page Logstash - OpenSearch documentation
We initially planned to use the Cross-cluster replication for Amazon OpenSearch Service - Amazon OpenSearch Service AWS provided cross cluster replication strategy through input/outbound(leader/follower) concept. But, we were been told that it will be mostly used for the DR purpose not for the live data sync between 2 domains in different accounts.
So, could you please suggest, how we can achieve this replication and enable the logstash input plugin on AWS OpenSearch 1.3 cluster?
AWS provided cross cluster replication is same feature for managed OpenSearch domains. It does provide live data syncing with an SLA of 1 minute lag. Feel free to use managed service feature for both live data syncing and disaster recovery.
Yeah, we are using the AWS provided OpenSearch CCR by enabling the Inbound/Outbound connections between the leader/follower domains. And, we were been told by AWS SME that it should be used only for the DR solution not for the live data sync as it takes some time to replicate between the two domains.
And, he also suggested logstash lambda approach for live data sync between 2 domains. So, we are exploring these options too though we didn’t see any issues with the above DR sync option.
Curious to know where you found the AWS provided SLS of 1 minute lag with provided CCR, would be great if you can share any page on that.