In the past we used curator to delete the indices within Open Distro via a rule:
indices: syslog-*
age: -60d
This rule was working as expected. The latest curator is not working anymore with the latest OpenSearch version: “Unable to connect to Elasticsearch cluster. Error: The client noticed that the server is not Elasticsearch and we do not support this unknown product”. So i take a close look at the OpenSearch documentation Policies - OpenSearch documentation
So my question is, what is the best/simplest way to delete those indices? The requirements are simple and not complex (without hot/cold state and so on…). The expamples in the documentation are little bit to much.
Another way which can be found is Amazon OpenSearch Service but as far as i can see it, this is deprecated.
Hey @thsul
That’s super interesting. So the problem you are facing is related to a license check being placed into elasticsearch-py (which is being used by curator) - more context on this issue: Carrying forward the OpenSearch client libraries as a community · OpenSearch . I’m currently digging in to see if there is a workaround for this use (first call out I’ve seen for it, tbh).
The link to the AWS service docs should work but you would need to substitute elasticsearch-py for opensearch-py. It’s a drop-in replacement, so it should get you going.
This would probably be a change to your workflow, but the Index Management features of OpenSearch are probably the best way to manage those indices as they age (and no extra scripting is required). It can seem daunting at first, but it’s super powerful.
thanks for the quick reply. I installed the curator via the pip3 tool. I will test the drop-in replacement, thanks once more!
Do you have some sample configs for the index management? The examples on the documentation are to specific and not apply on the simple date value. I agree that the index management is super powerful and it seems, that this is the better solution compared with curator.