Data prepper 2.10.2
Creating @timestamp from Syslog time format using this processor:
- date:
match:
- key: time
patterns: ["MMM dd HH:mm:ss"]
‘time’ field comes from Fluentbit standard syslog parser.
Example time field:
"time":"Apr 4 13:17:07"
When day number is one digit like in the above example Data prepper will not create @timestamp field.
When day number is two digit:
"time":"Mar 31 16:59:10"
date processor matches and @timefield is added to the index.
The reason looks obvious:
"time":"Apr 4 13:17:07"
"time":"Mar 31 16:59:10"
two spaces vs one space between month and day. And this fails the dataprepper match for date pattern.
When I change the processor to this:
- date:
match:
- key: time
patterns: ["MMM dd HH:mm:ss"]
It works (added extra space). But that is not a solution of course. What am I missing here? Because I simply don’t believe that Dataprepper cannot handle the most popular date format in Linux.
P.