# \[Solved\] Fluentd/Opensearch - How to ingest nested types?

**URL:** <https://forum.opensearch.org/t/solved-fluentd-opensearch-how-to-ingest-nested-types/10203>\
**Category:** OpenSearch\
**Created:** [July 7, 2022, 8:40am UTC](https://forum.opensearch.org/t/solved-fluentd-opensearch-how-to-ingest-nested-types/10203 "2022-07-07T08:40:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wasabi](https://avatars.discourse-cdn.com/v4/letter/w/c5a1d2/32.png) [@Wasabi](https://forum.opensearch.org/u/Wasabi)\
**Post date:** [July 7, 2022, 8:40am UTC](https://forum.opensearch.org/t/solved-fluentd-opensearch-how-to-ingest-nested-types/10203/1 "2022-07-07T08:40:54Z")

</div>

I have configured a index to contain a nested field type, such that we can later do more complex queries on the nested field. I basically used:

```auto
PUT app.backend.app_logs
{
  "mappings": {
    "properties": {
      "related_objects": {
        "type": "nested" 
      }
    }
  }
}

```

My problem now is that I cannot send logs from fluentd to opensearch anymore, as opensearch rejects the log messages. I assume that’s because fluentd has field called “related\_objects” but that’s just a (JSON) string.

Example log:

`fluentd | 2022-07-07 10:28:14 +0200 [warn]: #0 dump an error event: error_class=Fluent::Plugin::OpenSearchErrorHandler::OpenSearchError error="400 - Rejected by OpenSearch" location=nil tag="filter.app.backend.app_logs" time=2022-07-07 10:27:05.310000000 +0200 record={"docker.container_id"=>"c3fd494fbd9e6f08dfa1e8bdbd57bf1a94422805f090141df7a94dece52fcfad", "time"=>"2022-07-07T08:27:05.310555466Z", "stream"=>"stdout", "docker.container_started"=>"2022-07-06T13:13:32.633261163Z", "docker.container_image"=>"image_path", "docker.container_name"=>"intranet-worker", "module"=>"org-worker", "severity"=>"DEBUG", "file"=>"logger.py", "function"=>"event_user_accepted_offer", "message"=>"Found 0 other registrations to be deleted", "related_objects"=>"{\"briefing_offer\": 24}", "target_index"=>"app.backend.app_logs"}`

My question is how to ingest the nested object from fluentd? I’m sure there is some fluentd configuration I need to do, but as far as I understood the documentation, there is no object data type (just array, but its not an array).

In case it matters, I’m using the opensearch output plugin with the following configuration:

```auto
<match **>
  @type opensearch
  target_index_key target_index
  host opensearch-node1
  include_timestamp true
  port 9200
  scheme https
  user fluentd
  password xxx
  ssl_verify false
  index_name xxx
  ca_file /fluentd/etc/ssl/ca.crt.pem
</match>

```

Any help would be appreciated.

---

<div class="post-metadata">

**Author:** ![Wasabi](https://avatars.discourse-cdn.com/v4/letter/w/c5a1d2/32.png) [@Wasabi](https://forum.opensearch.org/u/Wasabi)\
**Post date:** [July 7, 2022, 10:16am UTC](https://forum.opensearch.org/t/solved-fluentd-opensearch-how-to-ingest-nested-types/10203/2 "2022-07-07T10:16:34Z")

</div>

The solution is actually so much simpler than I thought. Having the original log parsed and having the JSON in a field as string, I just need to parse this field and add the nested content back to it:

```auto
# Parse nested data in backend logs
<filter filter.app.backend.app_logs>
  @type parser
  key_name related_objects
  hash_value_field related_objects
  reserve_data true
  reserve_time true
  <parse>
    @type json
  </parse>
</filter>

```

That did the trick, now the logs are accepted again and I can do nested queries as expected.
