# Cross-Cluster Replication via NAT in OpenSearch 2.17.1 **URL:** \ **Category:** Cross-Cluster Replication\ **Tags:** troubleshoot\ **Created:** [May 26, 2025, 7:40am UTC](https://forum.opensearch.org/t/cross-cluster-replication-via-nat-in-opensearch-2-17-1/24520 "2025-05-26T07:40:55Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![vnovotny98](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vnovotny98](https://forum.opensearch.org/u/vnovotny98)\ **Post date:** [May 26, 2025, 7:40am UTC](https://forum.opensearch.org/t/cross-cluster-replication-via-nat-in-opensearch-2-17-1/24520/1 "2025-05-26T07:40:55Z")
**Version** 2.17.1 Plugins installed: all-nodes opensearch-alerting 2.17.1.0 all-nodes opensearch-anomaly-detection 2.17.1.0 all-nodes opensearch-asynchronous-search 2.17.1.0 all-nodes opensearch-cross-cluster-replication 2.17.1.0 all-nodes opensearch-custom-codecs 2.17.1.0 all-nodes opensearch-flow-framework 2.17.1.0 all-nodes opensearch-geospatial 2.17.1.0 all-nodes opensearch-index-management 2.17.1.0 all-nodes opensearch-job-scheduler 2.17.1.0 all-nodes opensearch-knn 2.17.1.0 all-nodes opensearch-ml 2.17.1.0 all-nodes opensearch-neural-search 2.17.1.0 all-nodes opensearch-notifications 2.17.1.0 all-nodes opensearch-notifications-core 2.17.1.0 all-nodes opensearch-observability 2.17.1.0 all-nodes opensearch-performance-analyzer 2.17.1.0 all-nodes opensearch-reports-scheduler 2.17.1.0 all-nodes opensearch-security 2.17.1.0 all-nodes opensearch-security-analytics 2.17.1.0 all-nodes opensearch-skills 2.17.1.0 all-nodes opensearch-sql 2.17.1.0 all-nodes opensearch-system-templates 2.17.1.0 all-nodes prometheus-exporter 2.17.1.0 all-nodes query-insights 2.17.1.0 **Dear OpenSearch Support,** We are experiencing issues with cross-cluster replication (CCR). We are currently on OpenSearch version 2.17.1. ### Cluster Setup We have two OpenSearch clusters: - **Primary cluster** : 15 nodes (3 master, 6 hot, 3 warm, 3 cold) - **Secondary cluster** : 5 nodes (1 master, 2 hot, 1 warm, 1 cold) Each group of node types is on its own VLAN (Master VLAN, Hot VLAN, etc.). The two clusters are permanently running and reachable over **NAT** , where: - Port `9200` and `9300` are open between corresponding node types: - NAT maps external IPs for primary nodes (e.g. `nat-master1-ip:9300`) - Nodes in both clusters use the **same internal IPs/hostnames** , so NAT is required to resolve between them. Each node is configured with the same `network.publish_host` and `network.host`. ### Problem Description We want to replicate data from the primary cluster to the secondary cluster. We created appropriate users and roles for replication. We attempted two configurations on the **secondary cluster** : #### 1. **Seed mode** ```auto PUT /_cluster/settings?pretty { "persistent" : { "cluster" : { "remote" : { "connection-to-primar" : { "seeds" : [ "nat-master1-ip:9300", "nat-master2-ip:9300", "nat-master3-ip:9300" ], "transport.compress": true } } } } } ``` → Result: ```auto "num_nodes_connected": 0, "max_connections_per_cluster": 3 ``` #### 2. **Proxy mode** ```auto PUT /_cluster/settings?pretty { "persistent": { "cluster": { "remote": { "connection-to-primar-proxy": { "mode": "proxy", "proxy_address": "nat-master1-ip:9300", "transport.compress": true } } } } } ``` → Result: ```auto "num_proxy_sockets_connected": 18, "max_proxy_socket_connections": 18 ``` With proxy mode, the connection appears to be established. ### Replication Attempt When we trigger replication, authentication passes, and in logs we see: - On **primary cluster** : ```auto Replication setup - Permissions validation successful for Index ``` - On **secondary cluster** : ```auto Failed to trigger replication for xxx-test-000006 - ResourceAlreadyExistsException[task with id {replication:index:xxx-test-000006} already exists] ``` However, the index is not created, and no data is being replicated. ### My Main Question **Do all nodes in the secondary cluster need to have direct access to `nat-master1-ip:9300` (or other seed/proxy nodes on the leader)?** Or is it sufficient that only the master nodes are connected? We appreciate your help and any clarification you can provide. ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/3bb315e7d378b871ee8e07ec44d01ea83c450907.png) Kind regards, Vojtech ---
**Author:** ![vnovotny98](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vnovotny98](https://forum.opensearch.org/u/vnovotny98)\ **Post date:** [May 28, 2025, 9:18am UTC](https://forum.opensearch.org/t/cross-cluster-replication-via-nat-in-opensearch-2-17-1/24520/2 "2025-05-28T09:18:28Z")
any advice? ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\ **Post date:** [July 27, 2025, 9:19am UTC](https://forum.opensearch.org/t/cross-cluster-replication-via-nat-in-opensearch-2-17-1/24520/3 "2025-07-27T09:19:19Z")
Closed on July 27, 2025, 9:19am UTC This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.