I am using this version of opensearch:
"version": {
"distribution": "opensearch",
"number": "1.3.11",
"build_type": "rpm",
"build_hash": "d104ed30b8c30d4e3313c30882ca8563feef5775",
"build_date": "2023-06-23T18:09:44.285311Z",
"build_snapshot": false,
"lucene_version": "8.10.1",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
I have the following mapping:
"properties": {
"DATE_CREATED": {
"type": "date",
"format": "strict_date_optional_time||epoch_second"
}
}
When I try to insert the following value into this index:
"DATE_CREATED": -2177434800
I get the following error:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-2177434800'"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-2177434800'",
"caused_by": {
"type": "date_time_exception",
"reason": "Invalid value for Year (valid values -999999999 - 999999999): -2177434800"
}
},
"status": 400
}
It looks like -2177434800
(Tuesday, 1 January 1901 05:00:00) is being converted using the strict_date_optional_time
(and failing) rather than the epoch_second
convertor.
I can fix this by changing the mapping from:
"format": "strict_date_optional_time||epoch_second"
to
"format": "epoch_second||strict_date_optional_time"
however this would require me to create a new index and re-index all my docs.
I think this is a bug as -2177434800
should not be trying to be converted using the strict_date_optional_time
convertor, or if it fails, then the epoch_second
convertor should be used?
I am also seeing slightly different errors with other date values such as -377476136
:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-377476136'"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [DATE_CREATED] of type [date] in document with id '2128'. Preview of field's value: '-377476136'",
"caused_by": {
"type": "arithmetic_exception",
"reason": "long overflow"
}
},
"status": 400
}
Thanks in advance.
Update: Seems like I’m hitting this issue: date mappings with strict_date_optional_time || epoch_millis behave inconsitently · Issue #51080 · elastic/elasticsearch · GitHub