Hi, I have a project that requires me to close day old indices and delete indices older than 3 days. Currently I have an Index that requires to be closed/deleted at a later date depending on a certain field i.e. issue end date.
I don’t know how to implement this using the min_index_age currently my index uses a time based index naming scheme where yyyy-mm-dd is appended to the supposed time the close/delete timer should start (in my case the issue end date is used).
My question is does the close/delete condition based the age of the index via its index name (postfix yyyy-mm-dd) like curator and etc? or is it basing on an invisible age which starts to tick after creation?
If so any suggestions for this?
Btw I’m new here so please bare with me.Thank you in advance!
The “min_index_age” is calculated from the index creation time internally.
Can you explain your use case a bit more if that’s not enough for you? I see you mentioned closing indices after 1 day, deleting after 3 days. But, then it seems like you also have some type of dynamic field used to trigger when this “starts”? But, also mentioned that you have the date in the index name which has to happen at index creation time, so is it already known ahead of time? Maybe an example would help me understand .
I have an index that stores a start date and end date fields.
I am using the end date field as the date in the name of the index i.e, index_name.yyyy.mm.dd (not the exact format)
So technically this index should not be closed after 1 day and deleted after 3 days of creation but after end date + N day for closing or deleting action.
We are doing this because we are using this index (index 1) to correlate with another index (index 2) for alerting. Example, if index 2 timestamp doesn’t fall between any start date and end date from index 1, then send an alert that index 2 has this kind of data in it.
In summary, we are monitoring login/logout logs of (index 1) and checking if access (index 2) is allowed or not. We can delete index 1 right after index creation but we needed index 2 to stick around until it’s no longer relevant.
A few more questions. Is the end date you’re using static or dynamic?
i.e. do you have something dynamic like:
index-abc-<30 days later>
index-def-<5 days later>
index-ghi-<17 days later>
where you want ISM to be able to parse dynamic dates (and then + N) from the index name to use in it’s condition checking.
Or is it like:
index-abc-<30 days later>
index-def-<30 days later>
index-ghi-<30 days later>
where you can have the policy itself written with “30 days” since it will always be 30 days. And then you want to be able to do + N on top of this?
Actually it should be:
index-abc-<yyyy.mm.dd>
where yyyy.mm.dd is the date where we should + N days to start checking its conditions/perform action.
not:
index-abc-
where X is the number of days to be added to N (N is the age/number of days to perform close/delete action).
But either way, it shouldn’t matter which of the two approach is available since we can implement what we need on both approaches.
Actually I could think of an approach to implement this but it would involve a third party software, having IM handle this would have been absolutely awesome.
So we definitely do not have a way to currently do this.
Seems like we need to be able to parse an arbitrary date from the index name to use in a few different places (transitions, rollover).
Could you create a GitHub issue on our GitHub repository so we can track this?
.