Hi, I want to configure opensearch cross cluster replication across two different k8s clusters. How can i achieve this and i am using the 2.6.0 version of Opensearch

OpenSearch
,
1

Opensearch cross cluster replication

2

Hi @zainakram,

Have you checked Docs?

Best,
mj

3

yes @Mantas I have gone through this documentation but I am unable to achieve the cross cluster replication on two k8s clusters.

4

Would you mind elaborate a bit more? Any errors? what exactly is not working, and what steps did you already take?

thanks,
mj

5

hi @Mantas following are my config files of opensearch for DC-1 and DC-2

DC-1
cat opensearch-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: opensearch-config
namespace: elk
data:
opensearch.yml: |
cluster.name: “dc1-cluster”
node.name: “opensearch-leader”
node.attr.zone: “dc1”
network.host: “0.0.0.0”
discovery.seed_hosts: [“10.142.0.3”]
cluster.initial_cluster_manager_nodes: [“opensearch-leader”]
cluster.remote.dc2-cluster.seeds: [“10.142.0.7:32088”]
http.port: 9200
transport.port: 9300

cat opensearch-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: opensearch-leader
labels:
app: opensearch-leader
spec:
replicas: 1 # Adjust for more nodes in the cluster
selector:
matchLabels:
app: opensearch-leader
template:
metadata:
labels:
app: opensearch-leader
spec:
containers:
- name: opensearch
image: opensearchproject/opensearch:2.6.0 # Docker image
env:
- name: plugins.security.disabled
value: “true” # Disable security plugin if not needed
ports:
- containerPort: 9200 # HTTP port
- containerPort: 9600 # Performance Analyzer port
- containerPort: 9300 # Transport port for CCR
volumeMounts:
- name: config-volume
mountPath: /usr/share/opensearch/config/opensearch.yml
subPath: opensearch.yml
resources:
limits:
memory: “2Gi”
cpu: “1000m”
requests:
memory: “1Gi”
cpu: “500m”
volumes:
- name: config-volume
configMap:
name: opensearch-config # Reference to the ConfigMap you created

cat opensearch-service.yaml
apiVersion: v1
kind: Service
metadata:
name: opensearch-leader-service
spec:
selector:
app: opensearch-leader
ports:
- name: http
protocol: TCP
port: 9200
targetPort: 9200
- name: transport
protocol: TCP
port: 9300
targetPort: 9300 # Expose the transport port for cross-cluster communication
type: NodePort # Changed from NodePort to LoadBalancer

DC-2

cat opensearch-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: opensearch-config
namespace: elk
data:
opensearch.yml: |
cluster.name: “dc2-cluster”
node.name: “opensearch-follower”
node.attr.zone: “dc2”
network.host: “0.0.0.0”
discovery.seed_hosts: [“10.142.0.3”, “10.142.0.7”]
cluster.initial_cluster_manager_nodes: [“opensearch-follower”]
cluster.remote.dc1-cluster.seeds: [“104.196.41.2:32462”]
http.port: 9200
transport.port: 9300

cat opensearch-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: opensearch-follower
labels:
app: opensearch-follower
spec:
replicas: 1 # Adjust for more nodes in the cluster
selector:
matchLabels:
app: opensearch-follower
template:
metadata:
labels:
app: opensearch-follower
spec:
containers:
- name: opensearch
image: opensearchproject/opensearch:2.6.0 # Docker image
env:
- name: plugins.security.disabled
value: “true” # Disable security plugin if not needed
ports:
- containerPort: 9200 # HTTP port
- containerPort: 9600 # Performance Analyzer port
- containerPort: 9300 # Transport port for CCR
volumeMounts:
- name: config-volume
mountPath: /usr/share/opensearch/config/opensearch.yml
subPath: opensearch.yml
resources:
limits:
memory: “2Gi”
cpu: “1000m”
requests:
memory: “1Gi”
cpu: “500m”
volumes:
- name: config-volume
configMap:
name: opensearch-config # Reference to the ConfigMap you created

cat opensearch-service.yaml
apiVersion: v1
kind: Service
metadata:
name: opensearch-follower-service
spec:
selector:
app: opensearch-follower
ports:
- name: http
protocol: TCP
port: 9200
targetPort: 9200
- name: transport
protocol: TCP
port: 9300
targetPort: 9300 # Expose the transport port for cross-cluster communication
type: NodePort # Or LoadBalancer depending on your setup

after applying these files i am getting the following error in the logs of Opensearch for both pods of DC-1 and DC-2

[2024-11-19T05:25:43,402][INFO ][o.o.t.TransportService ] [opensearch-follower] publish_address {192.168.217.109:9300}, bound_addresses {[::]:9300}
[2024-11-19T05:26:13,443][WARN ][o.o.t.RemoteClusterService] [opensearch-follower] failed to connect to remote clusters within 30s
[2024-11-19T05:26:13,503][WARN ][o.o.t.SniffConnectionStrategy] [opensearch-follower] fetching nodes from external cluster [dc1-cluster] failed
org.opensearch.transport.ConnectTransportException: [104.196.41.2:32462] connect_timeout[30s]
at org.opensearch.transport.TcpTransport$ChannelsConnectedListener.onTimeout(TcpTransport.java:1082) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747) ~[opensearch-2.6.0.jar:2.6.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]