How to delete orphaned data

Plugins Index Management
1

I have cloned an ELK cluster with all its data and configurations. I recreated the cluster with the command “elasticsearch-node unsafe-bootstrap”, and I applied the following command to the other node to take it out of the old cluster and put it into the new one “elasticsearch-node detach-cluster”. I also recreated the security index with the new settings with the “securityadmin.sh” script.

Up to this point everything went fine. I have the new cluster up and running with no errors.

When I log into Kibana, I can see the new indexes correctly.

GET /_cat/indices?v
image

The problem is that all the data from the previous cluster is still there, taking up unnecessary space. The new cluster doesn’t see that data, at least the index data.

GET /_cat/allocation?v&pretty
shards disk.indices disk.used disk.avail disk.total
6 337kb 32,4gb 10,5gb 49,9gb
6 1.7mb 16,2gb 26,7gb 42,9gb

Notice that in image one, the indexes that ELK has occupy less than 1Mb, however, in this directory there are more than 30GB occupying unnecessary space.

How can I clean up the data that ELK is not using? I don’t need it at all. As far as I’m concerned they could be deleted without any problems.

Thanks

2

Try to find the folders under which all “old” indices are resided. They should be under your elasticsearch home folder/data/nodes/node index (0,1 etc.)/indices. You should see subfolders named as your indices. Detect those which your new cluster does not see and you don’t need them anymore and delete these folders.
If you indeed “detached” those old and unused indices from the new cluster, it should work. As you actually are going to make a rude action for deleting Elasticsearch internal files and folders, I strongly recommend to backup what you are going to delete, so you are able to restore the stuff just in case the new cluster still has references to them somehow.