Versions 2.9.0
Is this something I do through dashboards or via API? I just need to be able to close indices that are between 4 and 2 weeks, and delete anything older 4 weeks.
Index policy has hot and cold replica shard count change example. In ElasticSearch I do these using curator, so not sure how it’s done in Opensearch
In OpenSearch you’d typically use Index State Management. It’s similar to Curator, only it runs in OpenSearch itself.
I was referring to it, do you have json example for it? Visual Editor is also confusing, not seeing specific to retention policy.
Hey @stecino
Policy’s are here
As for Visual Editor you need to scroll down to “States” and then create your retention.
So went through the visual editor and created a policy, but I am not able to connect the states to each other. Can I get some pointers? This is what I have for my generated JSON
{
"id": "2_week_retention",
"seqNo": 0,
"primaryTerm": 1,
"policy": {
"policy_id": "2_week_retention",
"description": "This will close indices older than 2 weeks. It will also delete indices that are older than 4 weeks",
"last_updated_time": 1695854815084,
"schema_version": 18,
"error_notification": null,
"default_state": "open_index",
"states": [
{
"name": "open_index",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"open": {}
}
],
"transitions": []
},
{
"name": "close_index",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"close": {}
}
],
"transitions": []
},
{
"name": "delete_index",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"awx-*"
],
"priority": 1,
"last_updated_time": 1695854815084
}
]
}
}
Hey @stecino
of course, I can show you what I did but think the best way without filling this post up with screen shots is to show you my JSON config once saved you can open the Visual editor.
This is a 3 day rotate and delete.
{
"policy": {
"policy_id": "Winlogbeat 3 retention",
"description": " rollover policy.",
"last_updated_time": 1691722505995,
"schema_version": 18,
"error_notification": {
"channel": {
"id": "qR9mVYgBRnO25hGO-KYg"
},
"message_template": {
"source": "",
"lang": "mustache"
}
},
"default_state": "current",
"states": [
{
"name": "current",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"open": {}
}
],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_index_age": "3d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"winlogbeat",
"winlogbeat-*",
"winlogbeat-%{+YYYY.MM.dd}"
],
"priority": 200,
"last_updated_time": 1681780567047
}
]
}
}
If you noticed I have three Index set patterns, I forgot which one worked 
Also the priority I set @ 200 for other reasons, you can increase it to fit your needs.
Great!! Thank you
How often do created policies run, I am assuming it’s based on priority. Is there somewhere where it is set, or I can set? Meaning to frequency number based on priority.
I noticed I can apply policy on the list of the indices, but I want the policy to run like a cron? So do I need to setup an API call that will be execute the policy?
Hey @stecino
Here on with cron in it, rotate everyday
{
"policy": {
"policy_id": "Security 3 retention",
"description": " rollover policy.",
"last_updated_time": 1689301463468,
"schema_version": 18,
"error_notification": {
"channel": {
"id": "qR9mVYgBRnO25hGO-KYg"
},
"message_template": {
"source": "",
"lang": "mustache"
}
},
"default_state": "current",
"states": [
{
"name": "current",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"open": {}
}
],
"transitions": [
{
"state_name": "current",
"conditions": {
"cron": {
"cron": {
"expression": "0 1 * * *",
"timezone": "America/Chicago"
}
}
}
},
{
"state_name": "delete",
"conditions": {
"min_index_age": "3d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"security",
"security-auditlog-*",
"security-auditlog-%{+YYYY.MM.dd}"
],
"priority": 100,
"last_updated_time": 1681780567047
}
]
}
}
@Gsmitt I have setup the retention policy similarly to you first policy, where I am deleting indices older than 14 days. I have manually applied the policy to pre-existing indices that match the pattern.
Does this mean that policy is going to delete these in the screenshot?
@Gsmitt policy has succesfully deleted my old indices. Thanks again for the help!! Much appreciated
One more question 
Granted most of my index patterns require 14d retention, is there a limit on patterns that I can include in the policy without possibly introducing excessive overhead? Although same retention rule, should I create multiple policies instead of one monolithic one?
Hey @stecino
Good question, I dont know.
If there different index patterns I would , just for the simple fact I can keep track/monitor of each Index set easier then grouping different index set to gether.
