Removing a node in a cluster?

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

2.11.1

Describe the issue:

I have 3 nodes in my cluster and I want to replace each node one by one with a new, more powerful node.

I found this API in ElasticSearch : Put shutdown API | Elasticsearch Guide [7.17] | Elastic

I didn’t find this endpoint in OpenSearch documentation, i try the call and i get a 405 error.

What is the recommended practice for deleting a node?

Thanks

2

Hello @Yathus,

I have not encountered your exact scenario yet but I think this article by Opster would help in your case: How to Remove a Node from an Elasticsearch Cluster

This uses the cluster.routing.allocation.exclude cluster setting to drain a node of its stored shards so you could shut down a node safely (Cluster settings - OpenSearch Documentation)

I have not tried this on my test-setup but I think the workflow would look roughly like this:

  1. Exclude the node from shard allocation
  2. Power off the node
  3. Upgrade to the more powerful HW/VM
  4. Install OpenSearch and join the cluster
  5. Enable shard allocation again
  6. Wait until cluster health is Green again and repeat with the remaining nodes

BR,
Andreas

3

Thanks Andreas, i followed your proces and it worked.

However, wouldn’t the API offered by ElasticSearch be welcome? I’m also thinking of node restart, for example.

4

Happy to hear that it worked @Yathus!

But yeah I agree, a similar API endpoint would definitely improve the workflow for your use-case - I just took a look at their github and it seems that there already exists an issue exactly for that: Shutdown API for shutting down nodes in a safe manner · Issue #1304 · opensearch-project/OpenSearch · GitHub

Also could you mark the previous answer as the solution? Just for this thread to be “done”.

BR,
Andreas