Dataprepper key_value processor

Data Prepper
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Data-prepper version: Latest, deployed with docker

Describe the issue:
Cant set destination option to null with key_value processor plugin. If I let it default it parses every field inside parsed_message (as expected), but when I set it to null, for every event consumed from source it generates different types of errors. Do I need to set more options when destination is null?

Configuration:

http-fortilogs-pipeline:
  workers: 8
  source:
    http:
      ssl: false
  processor:
    - key_value:
        destination: null
        value_split_characters: "="
        field_split_characters: " "
    - delete_entries:
        with_keys: ["date","ident","pid","msgid","extradata","message"]
  sink:
    - stdout:

Relevant Logs or Screenshots:

2023-12-28T15:08:42,094 [http-fortilogs-pipeline-processor-worker-1-thread-4] ERROR org.opensearch.dataprepper.pipeline.ProcessWorker - Encountered exception during pipeline http-fortilogs-pipeline processing
java.lang.IllegalArgumentException: key States" must contain only alphanumeric chars with .-_ and must follow JsonPointer (ie. 'field/to/key')
        at org.opensearch.dataprepper.model.event.JacksonEvent.checkKey(JacksonEvent.java:411) ~[data-prepper-api-2.6.1.jar:?]
        at org.opensearch.dataprepper.model.event.JacksonEvent.checkAndTrimKey(JacksonEvent.java:400) ~[data-prepper-api-2.6.1.jar:?]
        at org.opensearch.dataprepper.model.event.JacksonEvent.put(JacksonEvent.java:139) ~[data-prepper-api-2.6.1.jar:?]
        at org.opensearch.dataprepper.plugins.processor.keyvalue.KeyValueProcessor.writeToRoot(KeyValueProcessor.java:530) ~[key-value-processor-2.6.1.jar:?]
        at org.opensearch.dataprepper.plugins.processor.keyvalue.KeyValueProcessor.doExecute(KeyValueProcessor.java:261) ~[key-value-processor-2.6.1.jar:?]
        at org.opensearch.dataprepper.model.processor.AbstractProcessor.lambda$execute$0(AbstractProcessor.java:54) ~[data-prepper-api-2.6.1.jar:?]
        at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:69) ~[micrometer-core-1.11.3.jar:1.11.3]
        at org.opensearch.dataprepper.model.processor.AbstractProcessor.execute(AbstractProcessor.java:54) ~[data-prepper-api-2.6.1.jar:?]
        at org.opensearch.dataprepper.pipeline.ProcessWorker.doRun(ProcessWorker.java:133) ~[data-prepper-core-2.6.1.jar:?]
        at org.opensearch.dataprepper.pipeline.ProcessWorker.run(ProcessWorker.java:61) [data-prepper-core-2.6.1.jar:?]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
2

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.