Create Repository is failing in multi mode setup

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 2.5.x (latest)

Describe the issue:
Create repository is failing in multi node setup and the same is working if we config as single node

Configuration:

version: ‘3’
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- path.repo=/usr/share/opensearch/snapshots
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- “OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m” # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
- opensearch-snapshot-data:/usr/share/opensearch/snapshots
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- path.repo=/usr/share/opensearch/snapshots
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- “OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m”
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
- opensearch-snapshot-data:/usr/share/opensearch/snapshots
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
ports:
- 5601:5601
expose:
- “5601”
environment:
OPENSEARCH_HOSTS: ‘[“https://opensearch-node1:9200”,“https://opensearch-node2:9200”]’
networks:
- opensearch-net

volumes:
opensearch-data1:
opensearch-data2:
opensearch-snapshot-data:

networks:
opensearch-net:

Relevant Logs or Screenshots:

Error:

{“error”:{“root_cause”:[{“type”:“repository_exception”,“reason”:“[testrepo] cannot create blob store”}],“type”:“repository_verification_exception”,“reason”:“[testrepo] path is not accessible on cluster-manager node”,“caused_by”:{“type”:“repository_exception”,“reason”:“[testrepo] cannot create blob store”,“caused_by”:{“type”:“access_denied_exception”,“reason”:“/usr/share/opensearch/snapshots/test_repo_location”}}},“status”:500}

Hey, && Welcome @Ramesh

I caught this in logs… I’m assuming thre might be a permission issue.

access_denied_exception”,“reason”:“/usr/share/opensearch/snapshots/test_repo_location”

&&

What do you see if you executed this?

curl -X GET localhost:9200/_cat/repositories?v=true

The API should return something liek this.

id    type
repo1   fs
repo2   s3
1 Like

Hi @Gsmitt , thanks for your reply,

I could see below response for

curl -X GET https://localhost:9200/_cat/repositories\?v\=true -ku 'admin:admin'

id type
testrepo fs