Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.16
Describe the issue:
I use vector.dev to index many thousands of messages per second. When ingesting journald logs we copy the journald __REALTIME_TIMESTAMP to @timestamp. However, one of many thousands of messages fail an insert with :
id":"lQ4IPpIB-9YBWDFhej6F","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"documents must contain a single-valued timestamp field ‘@timestamp’ of date type"}
This appears to be very difficult to track down in vector.dev as all tests against the timestamp being valid and existing before the post pass.
is it possible to enable extra logging in opensearch for it to log the message it is rejecting to index without a major impact on the ingestion rate.
The failed check appears to be in package org.opensearch.index.mapper