# Dynamic Index Creation based on document field

**URL:** <https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184>\
**Category:** OpenSearch\
**Created:** [July 8, 2024, 2:44pm UTC](https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184 "2024-07-08T14:44:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ashish25](https://avatars.discourse-cdn.com/v4/letter/a/67e7ee/32.png) [@Ashish25](https://forum.opensearch.org/u/Ashish25)\
**Post date:** [July 8, 2024, 2:44pm UTC](https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184/1 "2024-07-08T14:44:13Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

**Describe the issue** :

Hi,

I have below document:  
{“transactionId”:“1234”,“transactionType”:“card”, “timestamp”: “2023-11-30T14:41:12”}

I want to insert this document in the index based on transactionType, for example above document will goto index **payment\_{transactionType}\_{timestamp in date format}** i.e., **payment\_card\_2024-11-30**

Please help here.

**Configuration** :

**Relevant Logs or Screenshots** :

---

<div class="post-metadata">

**Author:** ![gaobinlong](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gaobinlong/32/5242_2.png) [@gaobinlong](https://forum.opensearch.org/u/gaobinlong)\
**Post date:** [July 9, 2024, 3:03am UTC](https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184/2 "2024-07-09T03:03:14Z")

</div>

You can use ingest pipeline to achieve that:

1. Create a ingest pipeline named `date-index-name`

```auto
PUT _ingest/pipeline/date_index_name
{
  "processors": [
      {
        "date_index_name": {
          "field": "timestamp",
          "index_name_prefix":"payment_{{transactionType}}_",
          "date_rounding" : "d",
          "date_formats": ["yyyy-MM-dd'T'HH:mm:ss"]
        }
      }
    ]
}

```

1. Index documents with the ingest pipeline

```auto
POST test1/_doc?pipeline=date_index_name
{
  "transactionId": "1234",
  "transactionType": "card",
  "timestamp": "2023-11-30T14:41:12"
}

```

1. Check the index and document:

```auto

GET payment_card_2023-11-30/_search

```

---

<div class="post-metadata">

**Author:** ![fizforeal15](https://avatars.discourse-cdn.com/v4/letter/f/f04885/32.png) [@fizforeal15](https://forum.opensearch.org/u/fizforeal15)\
**Post date:** [February 28, 2025, 1:10pm UTC](https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184/3 "2025-02-28T13:10:49Z")

</div>

Has anyone confirmed this works? I can’t get it to work in my open search cluster… here’s my pipeline configuration:

“processors”: [  
{  
“script”: {  
“lang”: “painless”,  
“source”: “if (ctx.retention == null) { ctx.retention = ‘R2Y’; }”  
}  
},  
{  
“date\_index\_name”: {  
“field”: “date”,  
“date\_formats”: [  
“ISO8601”  
],  
“index\_name\_prefix”: “logs\_{{retention}}\_”,  
“date\_rounding”: “M”,  
“index\_name\_format”: “yyyy-MM”,  
“ignore\_failure”: false  
}  
}  
]

I tested successfully that the “retention” field is added the the document before i added the template snippet to the index\_name\_prefix… after adding the template snippet, no new indexes are created and also I don’t see any failures in the processor stats:

{  
“date\_index\_name” : {  
“type” : “date\_index\_name”,  
“stats” : {  
“count” : 245,  
“time\_in\_millis” : 0,  
“current” : 0,  
“failed” : 0  
}  
}

changing index\_name\_prefix to a static string again fixes it… seems like the template snippets dont really work there?

---

<div class="post-metadata">

**Author:** ![fizforeal15](https://avatars.discourse-cdn.com/v4/letter/f/f04885/32.png) [@fizforeal15](https://forum.opensearch.org/u/fizforeal15)\
**Post date:** [March 7, 2025, 9:10am UTC](https://forum.opensearch.org/t/dynamic-index-creation-based-on-document-field/20184/4 "2025-03-07T09:10:42Z")

</div>

I’ve got it working! The index wasn’t being created because the value had uppercase letters.