Error whle starting data prepper org.opensearch.OpenSearchStatusException: Unable to parse response body

Data Prepper
1

Hello,

I am getting below error while starting data prepper Building [opensearch] as sink component 2022-03-11T06:04:17,781 [main] INFO com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Starting OpenSearch sink 2022-03-11T06:04:17,782 [main] INFO com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy 2022-03-11T06:04:17,932 [main] ERROR com.amazon.dataprepper.plugins.PluginFactory - Encountered exception while instantiating the plugin OpenSearchSink java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) [?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) [?:?] at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1] Caused by: org.opensearch.OpenSearchStatusException: Unable to parse response body at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1653) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1407) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] ... 19 more Suppressed: java.lang.IllegalStateException: Unsupported Content-Type: text/plain; charset=UTF-8 at org.opensearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1670) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1650) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1407) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) [?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) [?:?] at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1] Caused by: org.opensearch.client.ResponseException: method [GET], host [https://localhost:9200], URI [/_cluster/settings?master_timeout=30s&include_defaults=true], status line [HTTP/1.1 401 Unauthorized] Unauthorized at org.opensearch.client.RestClient.convertResponse(RestClient.java:339) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestClient.performRequest(RestClient.java:309) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] ... 19 more 2022-03-11T06:04:17,939 [main] ERROR com.amazon.dataprepper.parser.PipelineParser - Construction of pipeline components failed, skipping building of pipeline [raw-pipeline] and its connected pipelines com.amazon.dataprepper.plugins.PluginException: Encountered exception while instantiating the plugin OpenSearchSink at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:45) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?] at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1] Caused by: java.lang.reflect.InvocationTargetException at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] ... 13 more Caused by: org.opensearch.OpenSearchStatusException: Unable to parse response body at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1653) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1407) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] ... 13 more Suppressed: java.lang.IllegalStateException: Unsupported Content-Type: text/plain; charset=UTF-8 at org.opensearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1670) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1650) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1407) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?] at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1] Caused by: org.opensearch.client.ResponseException: method [GET], host [https://localhost:9200], URI [/_cluster/settings?master_timeout=30s&include_defaults=true], status line [HTTP/1.1 401 Unauthorized] Unauthorized at org.opensearch.client.RestClient.convertResponse(RestClient.java:339) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestClient.performRequest(RestClient.java:309) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1] at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1] at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?] at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1] ... 13 more

I have my opensearch sink and collector running locally

pipelne.yaml

entry-pipeline:
  delay: "100"
  source:
    otel_trace_source:
      health_check_service: true
      ssl: false
  sink:
    - pipeline:
        name: "raw-pipeline"
    - pipeline:
        name: "service-map-pipeline"
raw-pipeline:
  source:
    pipeline:
      name: "entry-pipeline"
  prepper:
    - otel_trace_raw_prepper:
  sink:
    - opensearch:
        hosts: 
          - "https://localhost:9200"
        insecure: true
        aws_region: "us-east-1"
        trace_analytics_raw: true
service-map-pipeline:
  delay: "100"
  source:
    pipeline:
      name: "entry-pipeline"
  prepper:
    - service_map_stateful:
  sink:
    - opensearch:
        hosts: 
          - "https://localhost:9200"
        username: "admin"
        password: "admin"
        insecure: true
        aws_region: "us-east-1"
        trace_analytics_service_map: true

otel-localconfig.yml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317
      http:
        endpoint: localhost:55681

processors:
  batch:

exporters:
  otlp/data-prepper:
    endpoint: localhost:21890
    insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/data-prepper]

Opensearch is also running locally and the dashboard is loading fine. Can anyone assist me with the error while starting data prepper

2

Hi,

Does anybody have any inputs on this? My data prepper is not able to parse the response body

3

Hi @ammujgd,

I am looking into the issue. Could you please provide the following:

  1. The version of Data Prepper you are using as well as the command to run it
  2. How is OpenSearch set up locally? Are you running through docker-compose? If so could you share the configuration?

Thanks!