How to do Multinode opensearch setup using docker compose by multiple machines :

OpenSearch
, ,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Describe the issue: i want to make Opensearch cluster using 3 machines in same network , what are the customizations should i do ,

Configuration:

version: '3'
services:
  opensearch-master1:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-master1
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-master1
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - cluster.initial_cluster_manager_nodes=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}

    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-master-data1:/usr/share/opensearch/data
        #- ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9201:9200
      - 9601:9600
    networks:
      - opensearch-net

  opensearch-master2:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-master2
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-master2
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - cluster.initial_cluster_manager_nodes=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-master-data2:/usr/share/opensearch/data
        # - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9202:9200
      - 9602:9600
    networks:
      - opensearch-net

  opensearch-master3:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-master3
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-master3
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - cluster.initial_cluster_manager_nodes=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-master-data3:/usr/share/opensearch/data
        # - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9203:9200
      - 9603:9600
    networks:
      - opensearch-net
  # Data Nodes
  opensearch-data1:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-data1
    environment:
      #- OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-data1
      - node.roles=data
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-data1:/usr/share/opensearch/data
        # - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9204:9200
      - 9604:9600
    networks:
      - opensearch-net

  opensearch-data2:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-data2
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-data2
      - node.roles=data
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-data2:/usr/share/opensearch/data
        # - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9205:9200
      - 9605:9600
    networks:
      - opensearch-net

  opensearch-data3:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-data3
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-data3
      - node.roles=data
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch_opensearch-data3:/usr/share/opensearch/data
        # - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9206:9200
      - 9606:9600
    networks:
      - opensearch-net



  opensearch-client1:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-client1
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster
      - node.name=opensearch-client1
      - node.roles=ingest,remote_cluster_client
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - cluster.initial_cluster_manager_nodes=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
          # volumes:
     # - opensearch-client-data1:/usr/share/opensearch/data
     #  - ./certs:/usr/share/opensearch/config/certs
    #  - ./opensearch-config/opensearch-client.yml:/usr/share/opensearch/config/opensearch.yml
    ports:
      - 9207:9200
      - 9607:9600
    networks:
      - opensearch-net

  opensearch-client2:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-client2
    environment:
      # - OPENSEARCH_SECURITY_ADMIN_PASSWORD=@Mine1623
      - cluster.name=opensearch-cluster 
      - node.name=opensearch-client2
      - node.roles=ingest,remote_cluster_client
      - discovery.seed_hosts=opensearch-master1,opensearch-master2,opensearch-master3
      - cluster.initial_cluster_manager_nodes=opensearch-master1,opensearch-master2,opensearch-master3
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - "DISABLE_INSTALL_DEMO_CONFIG=true"
      - "DISABLE_SECURITY_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
          #   volumes:
     # - opensearch-client-data2:/usr/share/opensearch/data
     #  - ./certs:/usr/share/opensearch/config/certs
    ports:
      - 9208:9200
      - 9608:9600
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:latest
    container_name: opensearch-dashboards
    environment:
     # - OPENSEARCH_PASSWORD=@Mine1623
      - 'OPENSEARCH_HOSTS=["http://opensearch-master1:9200","http://opensearch-master2:9200","http://opensearch-master3:9200"]'
      - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true"  # Disable security plugin
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ports:
      - 5601:5601
    networks:
      - opensearch-net
#environment:
  #- OPENSEARCH_SECURITY_ADMIN_PASSWORD=@VMware12

volumes:
  opensearch_opensearch-master-data1:
  opensearch_opensearch-master-data2:
  opensearch_opensearch-master-data3:
  opensearch_opensearch-data1:
  opensearch_opensearch-data2:
  opensearch_opensearch-data3:
 # opensearch-client-data1:
  #opensearch-client-data2:


networks:
  opensearch-net:

Relevant Logs or Screenshots:

image
image1366×768 243 KB

nodes are not aware of each other because 3 are in 1 machine , 3 are in 2nd machine and 3 are in other .

2

@Govind12 I’ve used your exact docker-compose and the cluster came up with no issues.

image
image1545×410 79.8 KB

image
image2002×204 115 KB

3

Yes its working , but i want to make a cluster by using 3 machines which are in same network , how these container will communicate to the containers , which are in different host machine??

i want that , 1 data node 1 master node and 1 client node in each machine and then they should make a cluster , also all the containers should visible in dashboard , i want to make highly available cluster .

4

@Govind12 In that case you’re talking about docker swarm which is not supported as far as I remember.
I suggest Kubernetes and Helm for a distributed environment.

1 Like
5

How to do data syncing in distributed environment ?

6

@Govind12 As you may know, data in OpenSearch are kept in shards that are placed on different Opensearch data nodes. All OpenSearch nodes have access to all data within the same cluster. OpenSearch is not aware of the underlying container infrastructure i.e. Docker, Kubernetes.
Distributed design regards OpenSearch pods which are managed by Kubernetes.

1 Like
7

would you recommend any reference for Setup of Opensearch cluster in K8s , a part from the official documentation ?

8

@Govind12 I’ve tested official OpenSearch charts with k3 Rancher and k8s (containerd, calico) on Ubuntu 22.04 LTS.
I had no issues with deployment.