Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
AWS Opensearch Service 2.5
Describe the issue:
I created rollup jobs for each index. We want to delete records older than 1 year from rollup indexes, it seems there is no way to do it (delete by query API doesn’t work on rollup indexes). Could you support on this request please?
Thanks
@davidev The delete by query API deletes only documents. Have you tried to use ISM with rollup indices?
What type of action should I use for this? If i delete the entire index i lose all the data. I just want to delete part of its data (records older than 1 year for example).
@davidev As far as I know it doesn’t delete all the indices but only the ones that meet the criteria of either size in MB, number of documents or age.
Take a look at this example.
You can create a test index and build the policy around it, then just manually trigger rollup on that index and test each stage. Please be aware that OpenSearch needs some time to process each state (hot, warm, cold, delete).
@davidev I’ve just noticed that you want to delete just the content.
The ISM will only manage the index. But if you’d like to delete the content (documents) then why would you like to keep the index?
Because I want to retain part of data (documents) of the rollup index. In this case last 1 year. This is to avoid that the size occupied is too much after more than a year.