ISM policy for deletion after a certain period of time

Open Source Elasticsearch and Kibana
1

I just want to delete the indexes directly after its min age but i don’t want index to go into hot or warm stage.

A nd also right now i have created a policy it will take more then the defined min age time to delete.

{
“policy_id”: “delete_after_2_days”,
“description”: “Policy that deletes indicies older than 20sec.”,
“last_updated_time”: 1650533329086,
“schema_version”: 1,
“error_notification”: null,
“default_state”: “close”,
“states”: [
{
“name”: “close”,
“actions”: [
{
“close”: {}
}
],
“transitions”: [
{
“state_name”: “delete”,
“conditions”: {
“min_index_age”: “10s”
}
}
]
},
{
“name”: “delete”,
“actions”: [
{
“delete”: {}
}
],
“transitions”:
}
],
“ism_template”: [
{
“index_patterns”: [
“yset-index-*”
],
“priority”: 0,
“last_updated_time”: 1650450452750
}
]
}

2

Hi Team ,

Can somebody help me here

3

you set min index age to 10 sec. I am not sure but task to check all policy conditions is not running every 10 sec. put more usable age, 5 min for instance

4

Thanks for reply ,
I have already test this with 1day min index age , also 5 min but still it takes more then 5 min to delete .

can you explain me how much time opensearch take to initialize any policy over an index pattern.

can we also the change the job_interval time in opensearch ?

5
6

Hi ,

I am not able to change the setting of cluster. Can you please check with this

PUT /_cluster/settings
{
“transient”: {
“opendistro.index_state_management.job_interval” : “5 Minute”
}
}

showing “Message”:“Your request: ‘/_cluster/settings’ payload is not allowed.”}

7

didn’t work with opensearch, mainly with opendistro. I remember that some of security settings requires to use client certificate:
curl --cert ./config/admin.pem --key ./config/admin.key -XPUT http://admin:password@localhost:9200/xxxxx
You also may change the settings in configuration file itself

8

Okay Thanks ,
One last question if i need to delete the previous index after somedays in opensearch when shards got max without taking care of time is it the right approach i am following???

{
“policy_id”: “delete_index_after_open_close_5mins”,
“description”: “Policy that deletes indicies older than 20sec.”,
“last_updated_time”: 1650970737786,
“schema_version”: 1,
“error_notification”: null,
“default_state”: “open”,
“states”: [
{
“name”: “open”,
“actions”: [
{
“open”: {}
}
],
“transitions”: [
{
“state_name”: “close”,
“conditions”: {
“min_index_age”: “5m”
}
}
]
},
{
“name”: “close”,
“actions”: [
{
“close”: {}
}
],
“transitions”: [
{
“state_name”: “delete”,
“conditions”: {
“min_index_age”: “5m”
}
}
]
},
{
“name”: “delete”,
“actions”: [
{
“delete”: {}
}
],
“transitions”:
}
],
“ism_template”: [
{
“index_patterns”: [
“shanky-index-*”
],
“priority”: 0,
“last_updated_time”: 1650970737786
}
]
}

or i need to update something in this policy

9

Your policy will delete index after 10 min (5 min in open state and 5 min in close). If you want to delete index based on number of docs or index size - use correspondent condition. Just remember that index will be deleted and new created if you do not use some sort of rollover parameters or index pattern naming

10

As per testing first it will take 30 min to initialized the policy after that it will lookover the min age i.e. 5 min after that it will again take 30 min to go into close stage also same for delete.

also we cannot change this 30min job interval