currently i have ism (index state management) that automatically rolls over at specified size\age setting and it then after configured number of days it is forced merged and moved to warm tier nodes and later at a configured date it is deleted.
can i add a condition such that warm transition which involves force merge happens only during off peak hours. i dont want to stop and start whole ism process… because i would like the rollover happening at scheduled time.
Hi @jathin12,
Yes, you can use the cron expression transition condition to set a specific time for the actions to execute.
As an example you use the condition to gate entry into a state
[cron expression @ 12AM] → (Some state with force merge ) → …rest of policy
The cron expression works by looking at the cron expression and seeing if the pattern has at least one match since the time it started checking the cron expression. This does mean though if you had it start checking at 1AM then it would not do it until the following 12AM in the above example, but you can always give a range or multiple transitions.
@dbbaughe – thanks for quick response… i was able to add temp-state in between hot and warm and the condition with corn… policy is updated.
now i try to update policy_seq_num on the existing indices using docs
i get a response stating the 9 indices are updated (which matches the count of indices in hot state)… but when i try to get ism policy attached to index using the api
GET _opendistro/_ism/explain/index_name
i am still getting old seq_num, which means it will transistion based on old policy, is that correct?
i need to know how to update seq_num of the policy attached to index.
Hey @jathin12,
I’m assuming you used the change_policy API to update the existing managed indices to the new updated policy right?
It is an asynchronous process that gets queued up on the job itself. If you view your .opendistro-ism-config index you should probably see the new policy_id stored in the change_policy field on the managed index job for the indices you applied it to.
It will only make the actual change once ISM views it as “safe” which is once the managed index reaches the end of the current state it’s in (or immediately if all you did was change a small configuration like min_doc_size 50 → 100).
Could you list your original policy, new policy and one of the explain responses for the index? I can try to see when it will change it.
current policy
{
“_id” : “logstash”,
“_version” : 51,
“_seq_no” : 132,
“_primary_term” : 2,
“policy” : {
“policy_id” : “logstash”,
“description” : “hot warm delete workflow”,
“last_updated_time” : 1611345959927,
“schema_version” : 1,
“error_notification” : null,
“default_state” : “hot”,
“states” : [
{
“name” : “hot”,
“actions” : [
{
“rollover” : {
“min_size” : “160gb”,
“min_index_age” : “5d”
}
}
],
“transitions” : [
{
“state_name” : “warm”,
“conditions” : {
“min_index_age” : “7d”
}
}
]
},
{
“name” : “warm”,
“actions” : [
{
“read_only” : { }
}
],
“transitions” : [
{
“state_name” : “ultrawarm”,
“conditions” : {
“cron” : {
“cron” : {
“expression” : “* 1-6 * * *”,
“timezone” : “America/New_York”
}
}
}
}
]
},
{
“name” : “ultrawarm”,
“actions” : [
{
“timeout” : “24h”,
“retry” : {
“count” : 5,
“backoff” : “exponential”,
“delay” : “1h”
},
“warm_migration” : { }
}
],
“transitions” : [
{
“state_name” : “delete”,
“conditions” : {
“min_index_age” : “40d”
}
}
]
},
{
“name” : “delete”,
“actions” : [
{
“delete” : { }
}
],
“transitions” :
}
]
}
}
i added warm step to add the cron condition, previously it just had hot, ultrawarm and delete.
it seems policy seq num on the index got updated to latest policy seq num. i guess it might just be timing which wasn’t obvious from the docs. i was expecting the ism policy seq num to be updated to lastest as soon as i try to run change_policy API. Your explanation makes sense now on why that’s happening.
thanks for explanation. i will check if ism is really waiting for off peak hours to migrate to ultra warm. if that does happen its a win.
change_policy docs needs some updates. it does not mention whats difference between state vs include.[n].state
@dbbaughe Hi Drew…
it seems i messed up… all the indices that were in hot\warm\ultrawam state got migrated to hot and waiting on roll over and then failing… i am trying to update them back to correct state and its not working. its stuck in hot state… need help here…
Hey @jathin12,
I’m guessing you used the “state” option in the API… this is a lack of documentation issue, I’m going to work with the technical writers to update it.
That “state” option lets you set the state the managed index should start in after the change policy takes affect. So if you had put “state”: “hot” then after the change policy happens it would move them all to the start of hot.
So immediate way to fix. Remove all your policies from affect indices. They will of course be in different lifecycles, so you need to create some temporary policies and reapply them to the indices. Temporary in the sense that they start in a start on the action where they left off from.
Edit: I have created a PR here: Updates ISM change policy API docs by dbbaughe · Pull Request #395 · opendistro/for-elasticsearch-docs · GitHub
Let me know if this clears up the API for you or if you think it needs more changes.
most of it looks good… but maybe you want differentiate between state and include.[0].state.
also is there a way to just include indices are not managed by any policy, so maybe null state?
Not sure I follow the “also is there a way to just include indices are not managed by any policy, so maybe null state?” part.
If the index is not being managed by ISM then this API is not used, you must use the AddPolicy API instead.
ok got it… thanks for clarification