# Failed to obtain node locks, tried \[\[/opt/opensearch/data\]\] with lock id \[0\] **URL:** \ **Category:** OpenSearch\ **Tags:** troubleshoot\ **Created:** [January 31, 2024, 11:23am UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660 "2024-01-31T11:23:49Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![YaswaniT](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/yaswanit/32/6708_2.png) [@YaswaniT](https://forum.opensearch.org/u/YaswaniT)\ **Post date:** [January 31, 2024, 11:23am UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/1 "2024-01-31T11:23:49Z")
**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser): 1.2.4 **Describe the issue** : I have installed opensearch in kubernetes cluster using our own helm chart. I have 2 data pods, 3 master pods and one ingest pod. I have done force deletion on all the pods by using “kubectl delete pods -n sample --all --force --grace-period=0”. After this, data pod is not ready even after 20 min. But getting error with lock id. Can anyone help to find the location of it and if there is anyway to remove the lock id without getting any error? **Relevant Logs or Screenshots** : “org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/opt/opensearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max\_local\_storage\_nodes] (was [1])?” ---
**Author:** ![gaobinlong](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gaobinlong/32/5242_2.png) [@gaobinlong](https://forum.opensearch.org/u/gaobinlong)\ **Post date:** [February 1, 2024, 2:07pm UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/2 "2024-02-01T14:07:36Z")
It seems that multiple nodes are using the same mounted volume, you check that configuration. ---
**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\ **Post date:** [February 1, 2024, 2:46pm UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/3 "2024-02-01T14:46:09Z")
@YaswaniT Did you delete the PVC after deleting the pods? ---
**Author:** ![YaswaniT](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/yaswanit/32/6708_2.png) [@YaswaniT](https://forum.opensearch.org/u/YaswaniT)\ **Post date:** [February 2, 2024, 5:48am UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/4 "2024-02-02T05:48:58Z")
No. I should not delete pvc. I have few indices in pvc. As part of the reboot, i had to force delete pods. Is there any other possible way to remove the lock? ---
**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\ **Post date:** [February 2, 2024, 3:52pm UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/5 "2024-02-02T15:52:59Z")
@YaswaniT Forcing the deletion of the pod and not allowing the service to stop gracefully could cause the reported issue. If you take a look at the data of the related PVC, you’ll find a file `node.lock` in `/nodes/0/` This file should be removed whenever you stop the service gracefully. Because you’ve forced it, the file remained. Do you have snapshots/backups of those indices? I’ve reproduced your issue. When I removed the `node.lock` file, the OpenSearch pod started again. ---
**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\ **Post date:** [February 2, 2024, 3:54pm UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/6 "2024-02-02T15:54:19Z")
@YaswaniT Why do you force deletion of the pod? Graceful deletion will also work and prevent node.lock issues. ---
**Author:** ![YaswaniT](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/yaswanit/32/6708_2.png) [@YaswaniT](https://forum.opensearch.org/u/YaswaniT)\ **Post date:** [February 6, 2024, 9:36am UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/7 "2024-02-06T09:36:59Z")
Hi @pablo , We have a testcase of force deleting the pods to check if it works without error. Now as this lock issue happened due to resilience issue(force delete or node down or power down), we are trying to find a solution. So by removing node.lock file from the “\/nodes/0/”, will the opensearch work? ---
**Author:** ![YaswaniT](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/yaswanit/32/6708_2.png) [@YaswaniT](https://forum.opensearch.org/u/YaswaniT)\ **Post date:** [February 6, 2024, 10:34am UTC](https://forum.opensearch.org/t/failed-to-obtain-node-locks-tried-opt-opensearch-data-with-lock-id-0/17660/8 "2024-02-06T10:34:07Z")
Hi @pablo , i removed the node.lock and i restarted the failure pod manually as it didnot restart on its own. Now i didnt get the issue. Thanks for your help.