Select_entries on nested objects

[andlyk]

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Version 2.0

Describe the issue:
I’m using the AWS ingestion pipeline (data prepper) and here I’m using the select_entries

select_entries

I want to select some objects which is nested and what I found in you docs is that nested object is handled by .dotting

Nested

However the field3.name and field4.adress (see configuration) is not selected and is not feed into the index.
Do you know why? And it could also be a AWS error but they link to your data prepper docs

Configuration:
The configuration looks like this:
processor:
- select_entries:
include_keys:
- field1
- field2
- field3.name
- field4.adress

Relevant Logs or Screenshots:

[Anthony]

@andlyk You would have to flatten the fields, see example below:

If you have the following document:

[
    {
      "field1": "abc",
      "field2": 123,
      "field3": {
        "name": "Alice",
        "surname": "Smith"
      },
      "field4": {
        "address": "123 Main St"
      }
    }
  ]'

You can use the follow pipeline configuration to extract the necessary fields.

processor:
    - flatten:
        source: field3
        target: ""     # Moves field3.name → name, field3.surname → surname
    - flatten:
        source: field4
        target: ""     # Moves field4.address → address
    - select_entries:
        include_keys:
          - field1
          - field2
          - name
          - surname
          - address

Result:

{"field1":"abc","address":"123 Main St","surname":"Smith","name":"Alice","field2":123}

[andlyk]

Hi Anthony!
Thank you for the reply!

I actually achieved the same result by using / (I found the suggestion here: Flatten nested maps - #3 by mbreb and tried it out)
So

• field1
• field2
• field3/name
• field4/adress

Works!
It could be nice if this was added to the documentation