Open Search Data Insert issue

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Good day, Team

I used helm charts to deploy opensearch in the Kubernetes cluster, and the storage is based on PVCs (persistent volume claims). I provided 20 GB of storage with 3 pods, and it is currently in running mode.
However, the problem is on the opensearch side, which means I am unable to store more than 82 lakh records because only 3.2 GB of the available 16.8 GB of space will be utilised.
Because the multi-node clusterer is returning the following error, i have utilised a single node here.

ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

some api reponse:

  1. settings api - {“projecttask”:{“settings”:{“index”:{“creation_date”:“1688036815059”,“number_of_shards”:“1”,“number_of_replicas”:“1”,“uuid”:“47vib6SvQi2jOB1MmsRmjg”,“version”:{“created”:“136287827”},“provided_name”:“projecttask”}}

  2. count api - HTTP/1.1 200 OK

content-type: application/json; charset=UTF-8

content-length: 77

{“count”:8194228,“_shards”:{“total”:1,“successful”:1,“skipped”:0,“failed”:0}}

  1. space allocation api - shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
    2 3.4gb 3.9gb 15.5gb 19.5gb 20 15.240.9.49 15.240.9.49 opensearch-cluster-master-0
    1 UNASSIGNED

Describe the issue:

Configuration:

Relevant Logs or Screenshots:

2

@ganeshsanpada You must set vm.max_map_count=262144 in all your k8s worker nodes.

1 Like
3

@pablo I appreciate the solution. I’ll try it out to see if it works.

4

@pablo Please advise me on the most effective way to set the vm.max_map_count=262144 in all worker nodes since whenever I try with a multi-node setup, I get problems with server uptime
ie. max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144].
Note : My requirement is to add more than 5 crore records.

5

sysctl -w vm.max_map_count=262144 should do. And then you’d want to put the same value in /etc/sysctl.conf