# Setup retention for indices **URL:** \ **Category:** OpenSearch\ **Created:** [September 25, 2023, 3:55pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042 "2023-09-25T15:55:25Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 25, 2023, 3:55pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/1 "2023-09-25T15:55:25Z")
**Versions** 2.9.0 Is this something I do through dashboards or via API? I just need to be able to close indices that are between 4 and 2 weeks, and delete anything older 4 weeks. Index policy has hot and cold replica shard count change example. In ElasticSearch I do these using curator, so not sure how it’s done in Opensearch ---
**Author:** ![radu.gheorghe](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/radu.gheorghe/32/3516_2.png) [@radu.gheorghe](https://forum.opensearch.org/u/radu.gheorghe)\ **Post date:** [September 26, 2023, 3:30pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/2 "2023-09-26T15:30:54Z")
In OpenSearch you’d typically use [Index State Management](https://opensearch.org/docs/latest/im-plugin/ism/index/). It’s similar to Curator, only it runs in OpenSearch itself. ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 26, 2023, 5:43pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/3 "2023-09-26T17:43:38Z")
![Screenshot 2023-09-26 at 10.41.13 AM](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/b/b1c0f7dc79026b01fc94182b43a5d3abab3bc153.png) I was referring to it, do you have json example for it? Visual Editor is also confusing, not seeing specific to retention policy. ---
**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\ **Post date:** [September 27, 2023, 4:33am UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/4 "2023-09-27T04:33:11Z")
Hey @stecino Policy’s are here > **[Policies](https://opensearch.org/docs/latest/im-plugin/ism/policies/)** > > Policies As for **Visual Editor** you need to scroll down to “States” and then create your retention. ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 27, 2023, 10:50pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/5 "2023-09-27T22:50:27Z")
So went through the visual editor and created a policy, but I am not able to connect the states to each other. Can I get some pointers? This is what I have for my generated JSON ```auto { "id": "2_week_retention", "seqNo": 0, "primaryTerm": 1, "policy": { "policy_id": "2_week_retention", "description": "This will close indices older than 2 weeks. It will also delete indices that are older than 4 weeks", "last_updated_time": 1695854815084, "schema_version": 18, "error_notification": null, "default_state": "open_index", "states": [ { "name": "open_index", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "open": {} } ], "transitions": [] }, { "name": "close_index", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "close": {} } ], "transitions": [] }, { "name": "delete_index", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "delete": {} } ], "transitions": [] } ], "ism_template": [ { "index_patterns": [ "awx-*" ], "priority": 1, "last_updated_time": 1695854815084 } ] } } ``` ---
**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\ **Post date:** [September 27, 2023, 11:10pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/6 "2023-09-27T23:10:48Z")
Hey @stecino > [@stecino](#): > > Can I get some pointers? of course, I can show you what I did but think the best way without filling this post up with screen shots is to show you my JSON config once saved you can open the Visual editor. This is a 3 day rotate and delete. ```auto { "policy": { "policy_id": "Winlogbeat 3 retention", "description": " rollover policy.", "last_updated_time": 1691722505995, "schema_version": 18, "error_notification": { "channel": { "id": "qR9mVYgBRnO25hGO-KYg" }, "message_template": { "source": "", "lang": "mustache" } }, "default_state": "current", "states": [ { "name": "current", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "open": {} } ], "transitions": [ { "state_name": "delete", "conditions": { "min_index_age": "3d" } } ] }, { "name": "delete", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "delete": {} } ], "transitions": [] } ], "ism_template": [ { "index_patterns": [ "winlogbeat", "winlogbeat-*", "winlogbeat-%{+YYYY.MM.dd}" ], "priority": 200, "last_updated_time": 1681780567047 } ] } } ``` If you noticed I have three Index set patterns, I forgot which one worked 😆 Also the priority I set @ 200 for other reasons, you can increase it to fit your needs. ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 27, 2023, 11:24pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/7 "2023-09-27T23:24:01Z")
Great!! Thank you ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 27, 2023, 11:29pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/8 "2023-09-27T23:29:56Z")
How often do created policies run, I am assuming it’s based on priority. Is there somewhere where it is set, or I can set? Meaning to frequency number based on priority. I noticed I can apply policy on the list of the indices, but I want the policy to run like a cron? So do I need to setup an API call that will be execute the policy? ---
**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\ **Post date:** [September 28, 2023, 1:04am UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/9 "2023-09-28T01:04:30Z")
Hey @stecino Here on with cron in it, rotate everyday ```auto { "policy": { "policy_id": "Security 3 retention", "description": " rollover policy.", "last_updated_time": 1689301463468, "schema_version": 18, "error_notification": { "channel": { "id": "qR9mVYgBRnO25hGO-KYg" }, "message_template": { "source": "", "lang": "mustache" } }, "default_state": "current", "states": [ { "name": "current", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "open": {} } ], "transitions": [ { "state_name": "current", "conditions": { "cron": { "cron": { "expression": "0 1 * * *", "timezone": "America/Chicago" } } } }, { "state_name": "delete", "conditions": { "min_index_age": "3d" } } ] }, { "name": "delete", "actions": [ { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" }, "delete": {} } ], "transitions": [] } ], "ism_template": [ { "index_patterns": [ "security", "security-auditlog-*", "security-auditlog-%{+YYYY.MM.dd}" ], "priority": 100, "last_updated_time": 1681780567047 } ] } } ``` ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 28, 2023, 3:56pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/10 "2023-09-28T15:56:44Z")
@Gsmitt I have setup the retention policy similarly to you first policy, where I am deleting indices older than 14 days. I have manually applied the policy to pre-existing indices that match the pattern. ![Screenshot 2023-09-28 at 8.54.26 AM](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/0/016d4acd2315d248bb33f1720f305b631a22cc45.png) Does this mean that policy is going to delete these in the screenshot? ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 28, 2023, 4:20pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/11 "2023-09-28T16:20:27Z")
@Gsmitt policy has succesfully deleted my old indices. Thanks again for the help!! Much appreciated ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://forum.opensearch.org/u/stecino)\ **Post date:** [September 28, 2023, 4:37pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/12 "2023-09-28T16:37:28Z")
One more question 😃 Granted most of my index patterns require 14d retention, is there a limit on patterns that I can include in the policy without possibly introducing excessive overhead? Although same retention rule, should I create multiple policies instead of one monolithic one? ---
**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\ **Post date:** [September 28, 2023, 9:05pm UTC](https://forum.opensearch.org/t/setup-retention-for-indices/16042/13 "2023-09-28T21:05:08Z")
Hey @stecino > [@stecino](#): > > is there a limit on patterns that I can include in the policy without possibly introducing excessive overhead Good question, I dont know. > [@stecino](#): > > should I create multiple policies instead of one monolithic one? If there different index patterns I would , just for the simple fact I can keep track/monitor of each Index set easier then grouping different index set to gether.