Having problems with ISM policies not being applied to the backing indices of datastreams

Plugins Index Management
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Opensearch 2.19

Describe the issue:

When an ISM policy exists and associated with a datastream index template, the newly created backing indices are not being managed by the policy

Configuration:

Relevant Logs or Screenshots:

I am seeing the same problem as outlines in ISM Policies does not take effect in DataStream indexes managed by index Template

It appears this issue was closed without being updated with a solution. Can someone tell me what is needed to fix the problem?

2

@CRRemacle Can you provide the commands you used to attach the policy to the data stream. I’ve tested using OS2.19.3 using the following and in my case the policy is applied to the backing indices:

1. Speed up ISM sweep to 1 minute:

curl -sku admin:'MyR3pr0@Secur3!' -XPUT https://localhost:9200/_cluster/settings \
  -H 'Content-Type: application/json' \
  -d '{"persistent": {"plugins.index_state_management.job_interval": 1}}'

2. Create the ISM policy (with ism_template matching the datastream name):

curl -sku admin:'MyR3pr0@Secur3!' -XPUT https://localhost:9200/_plugins/_ism/policies/my-ds-policy \
  -H 'Content-Type: application/json' \
  -d '{
    "policy": {
      "description": "manage my-logs-* datastreams",
      "default_state": "hot",
      "ism_template": [{"index_patterns": ["my-logs*"], "priority": 100}],
      "states": [{
        "name": "hot",
        "actions": [{"rollover": {"min_doc_count": 2}}],
        "transitions": []
      }]
    }
  }'

3. Create the index template (with data_stream: {}, no policy_id in settings):

curl -sku admin:'MyR3pr0@Secur3!' -XPUT https://localhost:9200/_index_template/my-logs-template \
  -H 'Content-Type: application/json' \
  -d '{
    "index_patterns": ["my-logs*"],
    "data_stream": {},
    "priority": 200
  }'

4. Create the datastream:

curl -sku admin:'MyR3pr0@Secur3!' -XPUT https://localhost:9200/_data_stream/my-logs-app

5. Index 2 documents to trigger the rollover:

curl -sku admin:'MyR3pr0@Secur3!' -XPOST https://localhost:9200/my-logs-app/_doc \
  -H 'Content-Type: application/json' \
  -d '{"@timestamp": "2026-05-28T10:00:00Z", "message": "doc 1"}'

curl -sku admin:'MyR3pr0@Secur3!' -XPOST https://localhost:9200/my-logs-app/_doc \
  -H 'Content-Type: application/json' \
  -d '{"@timestamp": "2026-05-28T10:00:01Z", "message": "doc 2"}'

After a couple of minutes the new backing index should also have policy id attached to it:

curl -sku admin:'MyR3pr0@Secur3!' "https://localhost:9200/_plugins/_ism/explain/.ds-my-logs-app-000002?pretty"

Result:
{
  ".ds-my-logs-app-000002" : {
    "index.plugins.index_state_management.policy_id" : "my-ds-policy",
    "index.opendistro.index_state_management.policy_id" : "my-ds-policy",
    "index" : ".ds-my-logs-app-000002",
    "index_uuid" : "BH7LqywcStSyNnf3TT00zA",
    "policy_id" : "my-ds-policy",
    "policy_seq_no" : 33,
    "policy_primary_term" : 1,
    "rolled_over" : false,
    "index_creation_date" : 1779957905792,
    "state" : {
      "name" : "hot",
      "start_time" : 1779957996336
    }, ...