How to Attach policies to indexes automatically in ILM

[divyank_1]

** Versions: 2.6.0** (relevant - Index Management Kibana):

Describe the issue:
I created an ILM policy for index security-auditlogs to purge index after 7 days.

ILM Policy created successfully:

PUT _plugins/_ism/policies/security-auditlogs
{
  "policy": {
    "description": "A simple default policy that delete index older than 7 days.",
    "default_state": "hot",
    "states":  [
            {
                "name": "hot",
                "actions": [],
                "transitions": [
                    {
                        "state_name": "delete",
                        "conditions": {
                            "min_index_age": "7d"
                        }
                    }
                ]
            },
            {
                "name": "delete",
                "actions": [
                    {
                        "delete": {}
                    }
                ],
                "transitions": []
            }
        ]
    "ism_template": {
    "index_patterns": ["security-auditlog-*"],
    "priority": 100
    }
  }
}

In Step 2: I don’t want to attach, the policy security_auditlogs to all the security index manually, It should get applied automatically but its not getting applied.

Pls let me know how to add ILM policy on indexes automatically.

Re link: Index State Management - OpenSearch documentation

Manully policy is getting added, but when new timeseries index will be created, the policy should get applied automatically.

[Gsmitt]

Hey @divyank_1

The ILM should pick it up in few minutes if it was created. For example I create one for 3 day retention using cron while ago. I forgot it about it and still working.

Security_ILM

{
    "id": "Security 3 retention",
    "seqNo": 273965,
    "primaryTerm": 6,
    "policy": {
        "policy_id": "Security 3 retention",
        "description": " rollover policy.",
        "last_updated_time": 1689301463468,
        "schema_version": 18,
        "error_notification": {
            "channel": {
                "id": "qR9mVYgBRnO25hGO-KYg"
            },
            "message_template": {
                "source": "",
                "lang": "mustache"
            }
        },
        "default_state": "current",
        "states": [
            {
                "name": "current",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "open": {}
                    }
                ],
                "transitions": [
                    {
                        "state_name": "current",
                        "conditions": {
                            "cron": {
                                "cron": {
                                    "expression": "0 1 * * *",
                                    "timezone": "America/Chicago"
                                }
                            }
                        }
                    },
                    {
                        "state_name": "delete",
                        "conditions": {
                            "min_index_age": "3d"
                        }
                    }
                ]
            },
            {
                "name": "delete",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "delete": {}
                    }
                ],
                "transitions": []
            }
        ],
        "ism_template": [
            {
                "index_patterns": [
                    "security",
                    "security-auditlog-*",
                    "security-auditlog-%{+YYYY.MM.dd}"
                ],
                "priority": 100,
                "last_updated_time": 1681780567047
            }
        ]
    }
}

Results

[divyank_1]

Now for new security-auditlog index policy is getting attached automatically.

I forgot to add ilm_template earlier and rollover alias in template.

    "ism_template": {
    "index_patterns": ["security-auditlog-*"],
    "priority": 100

PUT _template/security_delete 
{
   "index_patterns" : [
      "security-auditlog-*"
    ],
 "settings" : {
      "index" : {
        "opendistro" : {
          "index_state_management" : {
            "policy_id" : "security_index_policy",
            "rollover_alias" : "security-auditlog"
          }
        }
      }
    },
    "mappings" : { },
    "aliases" : {
      "security-auditlog" : { }
    }
  }

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "security-auditlog-*",
        "alias": "security-auditlog"
      }
    }
  ]
}