Setting index.mapping.total_fields.limit to a higher value each time a new index is created

General Feedback
,
1

Hi,

we are running OpenSearch 1.1.

At the beginning of every month a new index gets created. We have a problem where the default 1000 fields per index is too low for our needs. We would like to set this value to 1500 and have this applied every month automatically.

From what I’ve read in the documentation it seemed like this could be done by altering the index template by adding:

"settings": {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "1500"
          }
        }
      }
    }

I did this with a test template and created an index based on it. When I call:

GET _index_template/jat-test-template

I can see the setting above is there.

But when I call:

GET jat_test_2022.09/_settings

I get this:

{
  "jat_test_2022.09" : {
    "settings" : {
      "index" : {
        "creation_date" : "1663248285232",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "Ulo8TrVnRmGAualmYReSLg",
        "version" : {
          "created" : "135227827"
        },
        "provided_name" : "jat_test_2022.09"
      }
    }
  }
}

On other indices where we’ve manually set index.mapping.total_fields.limit to 1500 via a PUT request, when we view the settings of an index we see:

{
  "application-2022.09" : {
    "settings" : {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "1500"
          }
        },
        "number_of_shards" : "5",
        "provided_name" : "application-2022.09",
        "creation_date" : "1661990400522",
        "number_of_replicas" : "1",
        "uuid" : "2cyG-1opTRi0si-2eRXtPQ",
        "version" : {
          "created" : "135227827"
        }
      }
    }
  }
}

So it looks to me like my test template has not been applied to my test index.

Can you please tell me what I need to do to increase the limit automatically if it can’t be done via an index template?

Thanks in advance.

2

The reason I had this problem was I created an index that didn’t match the pattern in the template. So setting this value in the index template does work.

3

@jat -

Thanks for sharing your solution. I’m curious - do you have a specific need to have dynamic mappings? You can tell OpenSearch to index but not store in the source. You’ve also got the option of declaring a static mapping that ignores fields that don’t have an existing mapping.

Nate