During dynamic mapping of text field only 500 fields are inserted instead of 1000 fields

Plugins Index Management
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Opensearch 2.11.1

Describe the issue:
For example ,When I insert 1000 Date fields dynamically I can insert 1000 date fields successfully. But when i insert 1000 text fields dynamically only 500 is inserted. I have observed if its string automatically mapping is created as text and keyword.How to restrict this behavior to text field only and insert 1000 fields.

Kindly explain the behavior.
Configuration:

“test3”: {
“aliases”: {},
“mappings”: {
“properties”: {
“field1”: {
“type”: “text”,
“fields”: {
“keyword”: {
“type”: “keyword”,
“ignore_above”: 256
}
}
},

Relevant Logs or Screenshots:

POST http://localhost:9200/testdate/_bulk

{“index”:{“_id”:1}}
{“date_field1”: “2022-01-01”}
…
{“index”:{“_id”:1}}
{“date_field1000”: “2022-01-01”}

GET http://localhost:9200/testdate
{
“testdate”: {
“aliases”: {},
“mappings”: {
“properties”: {
“date_field1”: {
“type”: “date”
},
“date_field10”: {
“type”: “date”
},
“date_field100”: {
“type”: “date”
},
“date_field1000”: {
“type”: “date”
}
}

When i insert 1000 text fields dynamically, I could insert only 500 fields only.
POST http://localhost:9200/testtext/_bulk

{“index”:{“_id”:1}}
{“field1”: “Sample Text 1”}
…
{“index”:{“_id”:1}}
{“field1000”: “Sample Text 1”}

GET http://localhost:9200/test1
“field500”: {
“type”: “text”,
“fields”: {
“keyword”: {
“type”: “keyword”,
“ignore_above”: 256
}
}

2

Seems that the mappings limit controlled by index.mapping.total_fields.limit which defaults to 1000 has been reached, you can define an index template for the index or set mappings when creating index, int the mappings section, map string to only text data type explicitly, like this:

PUT my-index-000001
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "text"
          }
        }
      }
    ]
  }
}
1 Like
3

Thanks @gaobinlong it worked.

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.