I want to rotate an index at midnight every day. I also want to be able to sometimes rotate that index at other times.
Using Logstash to write to an index with the date as part of the name would work 90% of the time, but prevent me from rotating the index off-schedule. That’s where having an index alias that points to something that can easily be changed comes in handy. Is there a way I can get the best of both worlds?
What are the “at other times”? Do they come before the midnight time or after?
You can definitely rotate every night at midnight (although it will most likely never be exactly at midnight, can be off by some minutes because of the job execution scheduling).
Other times are when I update the index schema.
The other hiccup I’m running into is when I use an index policy to rotate an index every 24 hours, if I manually rotate the index, I now have 2 indices that now want to rotate every 24 hours.
When using ISM for rotating your indices (using rollover action) then yes, there was an issue where ISM would not play well when the user manually rotates the index out of band as ISM will continue to try and rollover the alias when it meets the conditions. I’m not sure what version you’re on, but in the recent OpenSearch versions there were changes to stop that from happening by failing the index if the alias doesn’t point to it as a write alias. Along with a more recent change to treat it as a success if you rollover the index yourself that should go out in 1.2.
As for your original problem of rolling over when you update the index schema… doesn’t sound like something we support.
Perhaps with the recent changes though, when you update the index schema you can rollover also and ISM will be bale to gracefully handle that situation now.
Good to know. I’m waiting for the Debian packages before I roll out Opensearch. I’m looking forward to that release.