# Problem with automatically adds policy

**URL:** https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475
**Category:** Index Management
**Created:** [March 30, 2021, 2:16am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475 "2021-03-30T02:16:33Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [March 30, 2021, 2:16am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/1 "2021-03-30T02:16:33Z")

</div>

Hi.  
I have a problem with automatic policy enforcement:  
If create a new policy specifying “ism\_template”  
{  
“policy”: {  
“policy\_id”: “autodel\_7days”,  
“description”: “delete index after 7 days”,  
“default\_state”: “norm”,  
“states”: [  
{  
“name”: “norm”,  
“actions”: ,  
“transitions”: [  
{  
“state\_name”: “del”,  
“conditions”: {  
“min\_index\_age”: “7d”  
}  
}  
]  
},  
{  
“name”: “del”,  
“actions”: [  
{  
“delete”: {}  
}  
],  
“transitions”:   
}  
],  
“ism\_template”: {  
“index\_patterns”: [  
“testindex\*”  
],  
“priority”: 50  
}  
}  
}

after creating a new index, the policy is not applied:  
PUT testindex1  
GET \_opendistro/\_ism/explain/testindex1

{  
“testindex1”: {  
“index.opendistro.index\_state\_management.policy\_id”: null  
}  
}

at the same time everything works as it should on the test cluster.  
production cluster contains 19 policies that are applied to 4000 indexes. 2 weeks ago, after the transition to 1.13.1, all policies were updated and for a while they were assigned normally. but now only old policies (created 2 weeks ago) are assigned, all newly created policies behave as in the example above.  
as if they do not perceive the “ism\_template” block.

Any ideas what could have gone wrong?

for templates “\_simulate\_index” (POST \_index\_template/\_simulate\_index/testindex) allows you to check which template will be assigned to the new index, maybe there is a similar command for checking the policy?

---

<div class="post-metadata">

### Author: ![dbbaughe](https://avatars.discourse-cdn.com/v4/letter/d/bbe5ce/32.png) [@dbbaughe](https://forum.opensearch.org/u/dbbaughe)
#### Post date: [March 31, 2021, 1:43am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/2 "2021-03-31T01:43:17Z")

</div>

Hi @mouse,

When you create an index that matches the index pattern do you see anything in the logs on the active master node? That is the one that should be parsing the cluster changed event and seeing there is a new index and trying to match it to a policy to create the managed index job.

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [March 31, 2021, 2:07am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/3 "2021-03-31T02:07:21Z")

</div>

@ [dbbaughe](https://forum.opensearch.org/u/dbbaughe)  
There are no error messages, I see only this:

[2021-03-31T04: 51: 07,094] [INFO] [c.a.o.j.s.JobSweeper] [vlg-elk-mst01] Running full sweep  
[2021-03-31T04: 53: 42,526] [INFO] [oecmMetadataCreateIndexService] [vlg-elk-mst01] [testindex99] creating index, cause [api], templates [mf\_default\_ism\_historyday], shards [1]/[one]  
[2021-03-31T04: 53: 47,291] [INFO] [oecraAllocationService] [vlg-elk-mst01] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[testindex99] [0]]]).  
[2021-03-31T04: 56: 07,094] [INFO] [c.a.o.j.s.JobSweeper] [vlg-elk-mst01] Running full sweep  
[2021-03-31T04: 56: 21,179] [INFO] [oecmMetadataCreateIndexService] [vlg-elk-mst01] [elk\_vault\_audit\_log-test-2021.03.30] creating index, cause [api], templates [mf\_vault], shards [1]/[one]  
[2021-03-31T04: 56: 21,773] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_vault\_audit\_log-test-2021.03.30] will be managed by policy [day\_90\_nodel]  
[2021-03-31T04: 56: 24.929] [INFO] [oecraAllocationService] [vlg-elk-mst01] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[elk\_vault\_audit\_log-test-2021.03 .30] [0]]]).

created two indices: testindex99 - falls under the pattern of the new policy (the message above), and elk\_vault\_audit\_log-test-2021.03.30 - under the existing one.

As a result of the tests, it turned out that if I now update the “day\_90\_nodel” policy, it will also stop being assigned automatically

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [March 31, 2021, 4:45am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/4 "2021-03-31T04:45:08Z")

</div>

It was not easy, but I solved the problem.  
I started to remove template assignment from all policies (set “ism\_template”: null)  
When there were only two politicians with templates left, they began to work out en masse. At the same time, I did not create new indexes. Log from the master:

[2021-03-31T07: 31: 07,107] [INFO] [c.a.o.j.s.JobSweeper] [vlg-elk-mst01] Running full sweep  
[2021-03-31T07: 32: 43,259] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-utaginfra-perfmon-2021.03.30] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-test-2] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_dc-logon-test-99] will be managed by policy [autodel\_7days]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [security-auditlog-2021.03.30] will be managed by policy [autodel\_7days]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-2021.03.31] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-test-1] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-test-3] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-utaginfra-perfmon-2021.03.31] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-2021.03.30] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [security-auditlog-2021.03.31] will be managed by policy [autodel\_7days]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_nfvi\_ethernet-2021.03.31] will be managed by policy [autodel\_7days]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_techresources-techxa-vda-perfmon-test-4] will be managed by policy [day\_60-30]  
[2021-03-31T07: 32: 43,260] [INFO] [c.a.o.i.i.ManagedIndexCoordinator] [vlg-elk-mst01] Index [elk\_nfvi\_ethernet-2021.03.30] will be managed by policy [autodel\_7days]

Probably there were some problems with the intersection of patterns, but no errors appeared when saving policies, and all policies were manually assigned to the indexes without errors.

UPD:  
I started restoring templates in policies anew, checking how it work after each and again stumbled upon the problem.  
It occurs on exactly 11 policy containing index templates. At the same time, there is no difference what kind of index pattern in 11 policy, it simply does not work.  
If I clear the pattern at 10 (set “ism\_template”: null) then 11 works without problems.  
The question arises: are there any limits on the number of policies containing index\_patterns?

---

<div class="post-metadata">

### Author: ![tony](https://avatars.discourse-cdn.com/v4/letter/t/ee7513/32.png) [@tony](https://forum.opensearch.org/u/tony)
#### Post date: [March 31, 2021, 7:37am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/5 "2021-03-31T07:37:46Z")

</div>

Is this a resource issue with your cluster? How many nodes on your production cluster?

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [March 31, 2021, 10:57am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/6 "2021-03-31T10:57:55Z")

</div>

@tony

my error is clearly reproduced on the production cluster (18 data nodes, 3 master, 6 ingest, 14 coordination. 12 cores CPU + 64 GB RAM for each) and on the test cluster (3 data nodes, 1 master, 2 ingest. 4 cores CPU +8 GB RAM for each) as follows:

The same policies are consistently created, those  
“policy\_1” … “policy\_11” with patterns “test\_1- \*” … “test\_11- \*”  
the text does not matter, for example:  
{  
“policy”: {  
“policy\_id”: “policy\_1”,  
“description”: “delete index after 7 days”,  
“default\_state”: “norm”,  
“states”: [  
{  
“name”: “norm”,  
“actions”: ,  
“transitions”: [  
{  
“state\_name”: “del”,  
“conditions”: {  
“min\_index\_age”: “7d”  
}  
}  
]  
},  
{  
“name”: “del”,  
“actions”: [  
{  
“delete”: {}  
}  
],  
“transitions”:   
}  
],  
“ism\_template”: {  
“index\_patterns”: [  
“test\_1- \*”  
],  
“priority”: 50  
}  
}  
}

after creating each policy, an index is created  
**PUT test\_X-1**  
For the first 10, the policy is assigned, for the 11, no. Moreover, if, after creating 11 policies, you change any of the previously created ones, they also cease to be assigned

{  
“ **test\_1-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_1** ”,  
“index”: “test\_1-1”,  
“index\_uuid”: “X1n6d3\_YSLymYWHJVqTBow”,  
“policy\_id”: “policy\_1”  
},  
“ **test\_2-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_2** ”,  
“index”: “test\_2-1”,  
“index\_uuid”: “h9olWdoPTYygyqjdlYVx2Q”,  
“policy\_id”: “policy\_2”  
},  
“ **test\_3-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_3** ”,  
“index”: “test\_3-1”,  
“index\_uuid”: “t\_Ejz\_kMSna69o1iC3-o0A”,  
“policy\_id”: “policy\_3”  
},  
“ **test\_4-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_4** ”,  
“index”: “test\_4-1”,  
“index\_uuid”: “q2WmtiqjQLysYZ-2miSXdQ”,  
“policy\_id”: “policy\_4”  
},  
“ **test\_5-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_5** ”,  
“index”: “test\_5-1”,  
“index\_uuid”: “VziB0jHET5OqeapZbuOBew”,  
“policy\_id”: “policy\_5”  
},  
“ **test\_6-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_6** ”,  
“index”: “test\_6-1”,  
“index\_uuid”: “Wn\_cwBCpR9KwtiIhK7kyuw”,  
“policy\_id”: “policy\_6”  
},  
“ **test\_7-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_7** ”,  
“index”: “test\_7-1”,  
“index\_uuid”: “jgyoskK6Ts6Da0lBZcUz8Q”,  
“policy\_id”: “policy\_7”  
},  
“ **test\_8-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_8** ”,  
“index”: “test\_8-1”,  
“index\_uuid”: “0jSSQiImRfejIJgNsFR88A”,  
“policy\_id”: “policy\_8”  
},  
“ **test\_9-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_9** ”,  
“index”: “test\_9-1”,  
“index\_uuid”: “tceow8\_ZQVCylAO-LNUrHA”,  
“policy\_id”: “policy\_9”  
},  
“ **test\_10-1** ”: {  
“index.opendistro.index\_state\_management.policy\_id”: “ **policy\_10** ”,  
“index”: “test\_10-1”,  
“index\_uuid”: “Nkwvq42USK60HLi0BbbP4A”,  
“policy\_id”: “policy\_10”  
},  
**“test\_11-1”: {  
“index.opendistro.index\_state\_management.policy\_id”: null  
}**  
}

---

<div class="post-metadata">

### Author: ![tony](https://avatars.discourse-cdn.com/v4/letter/t/ee7513/32.png) [@tony](https://forum.opensearch.org/u/tony)
#### Post date: [March 31, 2021, 12:24pm UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/7 "2021-03-31T12:24:50Z")

</div>

@mouse I was more checking it wasn’t some kind of sharding issue.

---

<div class="post-metadata">

### Author: ![tony](https://avatars.discourse-cdn.com/v4/letter/t/ee7513/32.png) [@tony](https://forum.opensearch.org/u/tony)
#### Post date: [March 31, 2021, 12:50pm UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/8 "2021-03-31T12:50:27Z")

</div>

I have just created 11 polices (additional to my existing 3 policies) and then created 11 indices with each using a different policy and all 11 indices had a policy set against them. I was testing on a dual node cluster using 1.13.1.

---

<div class="post-metadata">

### Author: ![dbbaughe](https://avatars.discourse-cdn.com/v4/letter/d/bbe5ce/32.png) [@dbbaughe](https://forum.opensearch.org/u/dbbaughe)
#### Post date: [March 31, 2021, 3:12pm UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/9 "2021-03-31T15:12:01Z")

</div>

@mouse There is no limit on the templates that I am aware of, will also pull in the eng who worked on it to check.

@bowenlan Can you review the thread and see if you have any idea on what could cause the issue? And maybe try to reproduce with the patterns @mouse listed and see?

---

<div class="post-metadata">

### Author: ![bowenlan](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@bowenlan](https://forum.opensearch.org/u/bowenlan)
#### Post date: [March 31, 2021, 5:26pm UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/10 "2021-03-31T17:26:56Z")

</div>

@mouse one thing I can think of is:  
before checking if the newly created index matches with any ISM template, we will first do a check between ISM template lastUpdatedTime and index creationDate [code](https://github.com/opendistro-for-elasticsearch/index-management/blob/main/src/main/kotlin/com/amazon/opendistroforelasticsearch/indexmanagement/indexstatemanagement/ISMTemplateService.kt#L56), if the template got updated after the index created, it will not be used to do match check on this index anymore.

you can check template lastUpdatedTime by doing a GET policy call.

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [April 1, 2021, 12:01am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/11 "2021-04-01T00:01:19Z")

</div>

@bowenlan  
I created policies yesterday

**GET \_opendistro/\_ism/policies/policy\_11**  
… _“last\_updated\_time” : 1617187160398_, …

> **all response**
>
> {  
> “\_id” : “policy\_11”,  
> “\_version” : 1,  
> “\_seq\_no” : 130,  
> “\_primary\_term” : 5,  
> “policy” : {  
> “policy\_id” : “policy\_11”,  
> “description” : “delete index after 7 days”,  
> “last\_updated\_time” : 1617187160398,  
> “schema\_version” : 1,  
> “error\_notification” : null,  
> “default\_state” : “norm”,  
> “states” : [  
> {  
> “name” : “norm”,  
> “actions” : ,  
> “transitions” : [  
> {  
> “state\_name” : “del”,  
> “conditions” : {  
> “min\_index\_age” : “7d”  
> }  
> }  
> ]  
> },  
> {  
> “name” : “del”,  
> “actions” : [  
> {  
> “delete” : { }  
> }  
> ],  
> “transitions” :   
> }  
> ],  
> “ism\_template” : {  
> “index\_patterns” : [  
> “test\_11-\*”  
> ],  
> “priority” : 50,  
> “last\_updated\_time” : 1616992631502  
> }  
> }  
> }

index just created  
**PUT test\_11-2**

**GET \_opendistro/\_ism/explain/test\_11-2**  
{  
“test\_11-2” : {  
“index.opendistro.index\_state\_management.policy\_id” : null  
}  
}

**GET test\_11-2/\_settings**  
{  
“test\_11-2” : {  
“settings” : {  
“index” : {  
**“creation\_date” : “1617233667816”,**  
“number\_of\_shards” : “1”,  
“number\_of\_replicas” : “1”,  
“uuid” : “0IztZX-USemwrB3KkIM9xA”,  
“version” : {  
“created” : “7100299”  
},  
“provided\_name” : “test\_11-2”  
}  
}  
}  
}

creation date of index more than “last\_updated\_time” of policy  
1617233667816 vs 1617187160398

It should also be noted that the same policy (policy\_11) starts working **without any changes** if I delete a block “ism\_template”: {“index\_patterns […]}” in any of the 10 previously created ones.

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [April 2, 2021, 1:13am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/12 "2021-04-02T01:13:09Z")

</div>

@bowenlan @dbbaughe  
Noticed one more feature, on some data nodes I constantly see a lot of such messages in the log

> **log**
>
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_telegraf-arm-2021.01.24  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_telegraf-arm-2021.01.24  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_telegraf-arm-2021.01.23  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_telegraf-arm-2021.01.23  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_vault\_audit\_log-2021.01.29  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_vault\_audit\_log-2021.01.29  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_techresources-utaginfra-eventlog-2021.01.24  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_techresources-utaginfra-eventlog-2021.01.23  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_techresources-utaginfra-eventlog-2021.01.23  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_telegraf-arm-2021.01.29  
> [2021-04-02T04:10:25,060][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_telegraf-arm-2021.01.29  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_telegraf-arm-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_techresources-utaginfra-eventlog-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_telegraf-linux-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_telegraf-arm-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_telegraf-linux-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_techresources-utaginfra-eventlog-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_syslog-cicd-lgs-2021.01.28  
> [2021-04-02T04:10:25,335][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Finished executing attempt\_transition for elk\_syslog-cicd-lgs-2021.01.28  
> [2021-04-02T04:10:25,623][INFO][c.a.o.i.i.ManagedIndexRunner] [vlg-elk-data01] Executing attempt\_transition for elk\_syslog-arm-2021.01.31

The interval of their appearance coincides with the option  
“opendistro.index\_state\_management.job\_interval”  
but there are nodes that contain the primary shards of indexies whith ISM policy but do not contain such messages in the log.  
The logging level on all nodes of the cluster is the same.

Can you share information in which case these messages are written to the log and should they be on all data nodes?

And, sorry, I didn’t write in the first post I am using ODFE 1.13.1 but this is not a clean install, it was constantly updated from one of the first versions (0.8.0 or 0.9.0). Maybe some artifacts have accumulated?

---

<div class="post-metadata">

### Author: ![dbbaughe](https://avatars.discourse-cdn.com/v4/letter/d/bbe5ce/32.png) [@dbbaughe](https://forum.opensearch.org/u/dbbaughe)
#### Post date: [April 2, 2021, 1:36am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/13 "2021-04-02T01:36:50Z")

</div>

Hi @mouse,

Those logs show up on the nodes executing the internal jobs that manage the individual indices you have applied a policy to. The way it is distributed across nodes is to take the set of nodes that contain a shard of the .opendistro-ism-config index and split the job executions across these nodes. So it’s perfectly fine to not see these logs on some data nodes.

" It should also be noted that the same policy (policy\_11) starts working **without any changes** if I delete a block “ism\_template”: {“index\_patterns […]}” in any of the 10 previously created ones."

This looks like the default search size of 10 documents. @bowenlan Is the internal search for finding the matching ISM templates not correctly getting all policies but instead only the top 10?

---

<div class="post-metadata">

### Author: ![dbbaughe](https://avatars.discourse-cdn.com/v4/letter/d/bbe5ce/32.png) [@dbbaughe](https://forum.opensearch.org/u/dbbaughe)
#### Post date: [April 2, 2021, 1:39am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/14 "2021-04-02T01:39:00Z")

</div>

I think that is the issue, the internal search for ISM templates (policies) is not specifying the size so it’s defaulting to only getting 10 of them.

> <https://github.com/opendistro-for-elasticsearch/index-management/blob/main/src/main/kotlin/com/amazon/opendistroforelasticsearch/indexmanagement/indexstatemanagement/ManagedIndexCoordinator.kt#L316-L322>

@mouse Thanks for finding this issue! Would you be able to create a bug report for us on [GitHub - opendistro-for-elasticsearch/index-management: 🗃 Open Distro Index Management](https://github.com/opendistro-for-elasticsearch/index-management)

---

<div class="post-metadata">

### Author: ![mouse](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mouse](https://forum.opensearch.org/u/mouse)
#### Post date: [April 2, 2021, 2:40am UTC](https://forum.opensearch.org/t/problem-with-automatically-adds-policy/5475/15 "2021-04-02T02:40:18Z")

</div>

@dbbaughe  
Done

> <https://github.com/opendistro-for-elasticsearch/index-management/issues/429>
>
> \*\*Describe the bug\*\*
> "index\_patterns" for ISM template for eleventh or more are… not applied
> Probably is the issue, the internal search for ISM templates (policies) is not specifying the size so it’s defaulting to only getting 10 of them.
> 
> \*\*Other plugins installed\*\*
> ODFE 1.13.1
> 
> \*\*To Reproduce\*\*
> The same policies are consistently created:
> “policy\_1” … “policy\_11” with patterns “test\_1- \*” … “test\_11- \*”
> the text does not matter, for example:
> \`{
> 
> “policy”: {
> “policy\_id”: “policy\_1”,
> “description”: “delete index after 7 days”,
> “default\_state”: “norm”,
> “states”: \[
> {“name”: “norm”,“actions”: ,“transitions”: \[{“state\_name”: “del”,“conditions”: {“min\_index\_age”: “7d”}}\]},
> {“name”: “del”,“actions”: \[{“delete”: {}}\],“transitions”: }
> \],
> “ism\_template”: {
> “index\_patterns”: \[“test\_1-\*”\],
> “priority”: 50
> }
> }
> }\`
> 
> after creating each policy, an index is created
> \*\*PUT test\_X-1\*\*
> For the first 10, the policy is assigned, for the 11, no. Moreover, if, after creating 11 policies, you change any of the previously created ones, they also cease to be assigned
> \*\*GET \_opendistro/\_ism/explain/test\_\*\*\*
> {
> “test\_1-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_1\*\*”,
> “index”: “test\_1-1”,
> “index\_uuid”: “X1n6d3\_YSLymYWHJVqTBow”,
> “policy\_id”: “policy\_1”
> },
> “test\_2-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_2\*\*”,
> “index”: “test\_2-1”,
> “index\_uuid”: “h9olWdoPTYygyqjdlYVx2Q”,
> “policy\_id”: “policy\_2”
> },
> “test\_3-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_3\*\*”,
> “index”: “test\_3-1”,
> “index\_uuid”: “t\_Ejz\_kMSna69o1iC3-o0A”,
> “policy\_id”: “policy\_3”
> },
> “test\_4-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_4\*\*”,
> “index”: “test\_4-1”,
> “index\_uuid”: “q2WmtiqjQLysYZ-2miSXdQ”,
> “policy\_id”: “policy\_4”
> },
> “test\_5-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_5\*\*”,
> “index”: “test\_5-1”,
> “index\_uuid”: “VziB0jHET5OqeapZbuOBew”,
> “policy\_id”: “policy\_5”
> },
> “test\_6-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_6\*\*”,
> “index”: “test\_6-1”,
> “index\_uuid”: “Wn\_cwBCpR9KwtiIhK7kyuw”,
> “policy\_id”: “policy\_6”
> },
> “test\_7-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_7\*\*”,
> “index”: “test\_7-1”,
> “index\_uuid”: “jgyoskK6Ts6Da0lBZcUz8Q”,
> “policy\_id”: “policy\_7”
> },
> “test\_8-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_8\*\*”,
> “index”: “test\_8-1”,
> “index\_uuid”: “0jSSQiImRfejIJgNsFR88A”,
> “policy\_id”: “policy\_8”
> },
> “test\_9-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_9\*\*”,
> “index”: “test\_9-1”,
> “index\_uuid”: “tceow8\_ZQVCylAO-LNUrHA”,
> “policy\_id”: “policy\_9”
> },
> “test\_10-1”: {
> “index.opendistro.index\_state\_management.policy\_id”: “\*\*policy\_10\*\*”,
> “index”: “test\_10-1”,
> “index\_uuid”: “Nkwvq42USK60HLi0BbbP4A”,
> “policy\_id”: “policy\_10”
> },
> "test\_11-1": {
> \*\*“index.opendistro.index\_state\_management.policy\_id”: null\*\*
> }
> }
> 
> Policy (policy\_11) starts working \*\*without any changes\*\* if I delete a block “ism\_template”: {“index\_patterns \[…\]}” in any of the 10 previously created ones.
> 
> \*\*Additional context\*\*
> preliminary discussion \[here\](https://discuss.opendistrocommunity.dev/t/problem-with-automatically-adds-policy/5475/12)
