Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Data-prepper version: Latest, deployed with docker
Describe the issue:
Cant set destination option to null with key_value processor plugin. If I let it default it parses every field inside parsed_message (as expected), but when I set it to null, for every event consumed from source it generates different types of errors. Do I need to set more options when destination is null?
Configuration:
http-fortilogs-pipeline:
workers: 8
source:
http:
ssl: false
processor:
- key_value:
destination: null
value_split_characters: "="
field_split_characters: " "
- delete_entries:
with_keys: ["date","ident","pid","msgid","extradata","message"]
sink:
- stdout:
Relevant Logs or Screenshots:
2023-12-28T15:08:42,094 [http-fortilogs-pipeline-processor-worker-1-thread-4] ERROR org.opensearch.dataprepper.pipeline.ProcessWorker - Encountered exception during pipeline http-fortilogs-pipeline processing
java.lang.IllegalArgumentException: key States" must contain only alphanumeric chars with .-_ and must follow JsonPointer (ie. 'field/to/key')
at org.opensearch.dataprepper.model.event.JacksonEvent.checkKey(JacksonEvent.java:411) ~[data-prepper-api-2.6.1.jar:?]
at org.opensearch.dataprepper.model.event.JacksonEvent.checkAndTrimKey(JacksonEvent.java:400) ~[data-prepper-api-2.6.1.jar:?]
at org.opensearch.dataprepper.model.event.JacksonEvent.put(JacksonEvent.java:139) ~[data-prepper-api-2.6.1.jar:?]
at org.opensearch.dataprepper.plugins.processor.keyvalue.KeyValueProcessor.writeToRoot(KeyValueProcessor.java:530) ~[key-value-processor-2.6.1.jar:?]
at org.opensearch.dataprepper.plugins.processor.keyvalue.KeyValueProcessor.doExecute(KeyValueProcessor.java:261) ~[key-value-processor-2.6.1.jar:?]
at org.opensearch.dataprepper.model.processor.AbstractProcessor.lambda$execute$0(AbstractProcessor.java:54) ~[data-prepper-api-2.6.1.jar:?]
at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:69) ~[micrometer-core-1.11.3.jar:1.11.3]
at org.opensearch.dataprepper.model.processor.AbstractProcessor.execute(AbstractProcessor.java:54) ~[data-prepper-api-2.6.1.jar:?]
at org.opensearch.dataprepper.pipeline.ProcessWorker.doRun(ProcessWorker.java:133) ~[data-prepper-core-2.6.1.jar:?]
at org.opensearch.dataprepper.pipeline.ProcessWorker.run(ProcessWorker.java:61) [data-prepper-core-2.6.1.jar:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]