Negative epoch values failing to get parsed using strict_date_optional_time || epoch_second

OpenSearch
1

I am using this version of opensearch:

  "version": {
    "distribution": "opensearch",
    "number": "1.3.11",
    "build_type": "rpm",
    "build_hash": "d104ed30b8c30d4e3313c30882ca8563feef5775",
    "build_date": "2023-06-23T18:09:44.285311Z",
    "build_snapshot": false,
    "lucene_version": "8.10.1",
    "minimum_wire_compatibility_version": "6.8.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
  },

I have the following mapping:

"properties": {
    "DATE_CREATED": {
      "type": "date",
      "format": "strict_date_optional_time||epoch_second"
    }
  }

When I try to insert the following value into this index:

"DATE_CREATED": -2177434800

I get the following error:

{
    "error": {
        "root_cause": [
            {
                "type": "mapper_parsing_exception",
                "reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-2177434800'"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-2177434800'",
        "caused_by": {
            "type": "date_time_exception",
            "reason": "Invalid value for Year (valid values -999999999 - 999999999): -2177434800"
        }
    },
    "status": 400
}

It looks like -2177434800 (Tuesday, 1 January 1901 05:00:00) is being converted using the strict_date_optional_time (and failing) rather than the epoch_second convertor.

I can fix this by changing the mapping from:

"format": "strict_date_optional_time||epoch_second"

to

"format": "epoch_second||strict_date_optional_time"

however this would require me to create a new index and re-index all my docs.

I think this is a bug as -2177434800 should not be trying to be converted using the strict_date_optional_time convertor, or if it fails, then the epoch_second convertor should be used?

I am also seeing slightly different errors with other date values such as -377476136:

{
    "error": {
        "root_cause": [
            {
                "type": "mapper_parsing_exception",
                "reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-377476136'"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-377476136'",
        "caused_by": {
            "type": "arithmetic_exception",
            "reason": "long overflow"
        }
    },
    "status": 400
}

Thanks in advance.

Update: Seems like I’m hitting this issue: date mappings with strict_date_optional_time || epoch_millis behave inconsitently · Issue #51080 · elastic/elasticsearch · GitHub

2

Hey @r-opensearch

Have you tried this?