[Data Prepper] - Renaming nested json keys

Data Prepper
1

Hi @all,

I was trying to parse a json log using data prepper processor called “rename_keys” but I found that it is not possible to have parsing implemented on nested json keys to some other nested json keys.

For example -

{
   "level1": {
         "level2": {
             "level3": {
                    "key": "value"
               }
           }
    }
}

I want to parse this log, in such a way that level1.level2.level3.key should rename to level1.level2_temp.level3.key something like that.

{
   "level1": {
         "level2_temp": {
             "level3": {
                    "key": "value"
               }
           }
    }
}

CC @graytaylor0

1 Like
2

Hi @mann Thanks for trying out Data Prepper. This can be achieved in 2 ways,

  1. Using substitute_string Processor - You can define a regex pattern using this processor to replace the matching string . I think this approach suits your use-case if you want to send the log as it is to sink after renaming value. substitute_string - OpenSearch documentation
  2. Using parse_json Processor - Using this you can extract the level2 as a separate field and apply rename_keys. With this approach you will lose the outer key when you extract the level2 in to separate field. rename_keys - OpenSearch documentation
    Config example:
- parse_json:
        source: "message"
        pointer: "level1/level2"
    - rename_keys:
        entries:
        - from_key: "level2"
          to_key: "level2_temp"

Output:

{"message":"{\"level1\": {\"level2\": {\"level3\": {\"key\":\"value\"}}}}","level2_temp":{"level3":{"key":"value"}}}
3

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.