# Parse\_json proccesor

**URL:** <https://forum.opensearch.org/t/parse-json-proccesor/21043>\
**Category:** Data Prepper\
**Created:** [August 31, 2024, 2:38am UTC](https://forum.opensearch.org/t/parse-json-proccesor/21043 "2024-08-31T02:38:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dank1](https://avatars.discourse-cdn.com/v4/letter/d/7c8e57/32.png) [@dank1](https://forum.opensearch.org/u/dank1)\
**Post date:** [August 31, 2024, 2:38am UTC](https://forum.opensearch.org/t/parse-json-proccesor/21043/1 "2024-08-31T02:38:30Z")

</div>

Hey all,

I’m having some trouble using the `parse_json` processor with my OpenSearch ingestion pipeline on AWS. Here’s the setup: I upload a file to S3, which triggers an SQS queue, and that message gets indexed in OpenSearch. Below is a sample of one line in my JSON file:

```json
{"driver": "max verstappen", "log": "{\"level\":\"info\",\"ts\":1724484885.6000538}"}

```

My goal is to parse the log field, which seemed straightforward—I thought I could just set

`source: log`

in the configuration. However, it’s not working as expected. I tried different configurations like the one below:

```auto
version: "2"
log-pipeline4-level-up:
  source:
    s3:
      notification_type: "sqs"
      codec:
        newline: null # This specifies that each line in the S3 object is a separate log entry
      sqs:
        queue_url: "https://sqs.us-east-1.amazonaws.com/1111111111111/jsonlLogsQueue"
      compression: "none"
      aws:
        region: "us-east-1"
        # IAM role that the pipeline assumes to read data from the queue. This role must be the same as the pipeline role.
        sts_role_arn: "arn:aws:iam::11111111111:role/OpenSearchIngestionRole"
  processor:
    - parse_json:
        destination: new
  sink:
    - opensearch:
        hosts: 
          - "https://vpc-opensearch-workshop.us-west-2.es.amazonaws.com"
        index: "my-index"
        aws:
          region: "us-east-1"
          sts_role_arn: "arn:aws:iam::11111111111:role/OpenSearchIngestionRole"

```

As a result, I managed to get the following document in OpenSearch:

```auto

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "my-index",
        "_id": "1843ppEBN3hgemr_fuL9",
        "_score": 1,
        "_source": {
          "message": """{"driver": "max verstappen", "log": "{\"level\":\"info\",\"ts\":1724484885.6000538}"}""",
          "s3": {
            "bucket": "dank-jsonl-logs",
            "key": "driver.jsonl"
          },
          "new": {
            "driver": "max verstappen",
            "log": """{"level":"info","ts":1724484885.6000538}"""
          }
        }
      }
    ]
  }
}

```

I want to extract the `log` value not as a string but as a nested JSON object. Could anyone recommend what I should put in the configuration to achieve this? Thanks in advance!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [October 30, 2024, 2:39am UTC](https://forum.opensearch.org/t/parse-json-proccesor/21043/2 "2024-10-30T02:39:27Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.