My data prepper has started but why is this curl command failing

ammujgd · March 15, 2022, 6:29am

I started data prepper locally using docker run and its started

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Try the new cross-platform PowerShell https://aka.ms/pscore6

PS C:\Users\AnjanaAsok> cd /
PS C:\> cd '.\Program Files\'
PS C:\Program Files> cd .\Docker\
PS C:\Program Files\Docker> cd .\cli-plugins\
PS C:\Program Files\Docker\cli-plugins> docker run --expose 4900-v `/c/program files/docker/cli-plugins/pipeline.yaml:/usr/share/data-prepper/pipelines.yaml' -v `/c/program files/docker/cli-plugins/data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml' --network "host" opensearchproject/data-prepper:latest
docker: invalid range format for --expose: 4900-v, error: strconv.ParseUint: parsing "v": invalid syntax.
See 'docker run --help'.
PS C:\Program Files\Docker\cli-plugins> docker run --expose 4900 -v `/c/program files/docker/cli-plugins/pipeline.yaml:/usr/share/data-prepper/pipelines.yaml' -v `/c/program files/docker/cli-plugins/data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml' --network "host" opensearchproject/data-prepper:latest
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2022-03-15T06:16:46,942 [main] INFO  com.amazon.dataprepper.pipeline.server.DataPrepperServer - Creating Data Prepper server without TLS
2022-03-15T06:16:46,958 [main] INFO        com.amazon.dataprepper.DataPrepper - Using /usr/share/data-prepper/pipelines.yaml configuration file
2022-03-15T06:16:46,982 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building pipeline [entry-pipeline] from provided configuration
2022-03-15T06:16:46,982 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [otel_trace_source] as source component for the pipeline [entry-pipeline]
2022-03-15T06:16:47,111 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [entry-pipeline]
2022-03-15T06:16:47,117 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building preppers for the pipeline [entry-pipeline]
2022-03-15T06:16:47,118 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [entry-pipeline]
2022-03-15T06:16:47,119 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as sink component
2022-03-15T06:16:47,120 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as sink component
2022-03-15T06:16:47,123 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building pipeline [service-map-pipeline] from provided configuration
2022-03-15T06:16:47,123 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as source component for the pipeline [service-map-pipeline]
2022-03-15T06:16:47,123 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [service-map-pipeline]
2022-03-15T06:16:47,125 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building preppers for the pipeline [service-map-pipeline]
2022-03-15T06:16:47,371 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [service-map-pipeline]
2022-03-15T06:16:47,371 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [opensearch] as sink component
2022-03-15T06:16:47,382 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Starting OpenSearch sink
2022-03-15T06:16:47,394 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2022-03-15T06:16:47,545 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2022-03-15T06:16:47,972 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Started OpenSearch sink
2022-03-15T06:16:47,973 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building pipeline [raw-pipeline] from provided configuration
2022-03-15T06:16:47,973 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as source component for the pipeline [raw-pipeline]
2022-03-15T06:16:47,973 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [raw-pipeline]
2022-03-15T06:16:47,976 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building preppers for the pipeline [raw-pipeline]
2022-03-15T06:16:47,980 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [raw-pipeline]
2022-03-15T06:16:47,980 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [opensearch] as sink component
2022-03-15T06:16:47,987 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Starting OpenSearch sink
2022-03-15T06:16:47,987 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2022-03-15T06:16:47,988 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2022-03-15T06:16:48,247 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Started OpenSearch sink
2022-03-15T06:16:48,249 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - Initiating pipeline execution
2022-03-15T06:16:49,002 [main] INFO  com.amazon.dataprepper.plugins.source.oteltrace.OTelTraceSource - Started otel_trace_source...
2022-03-15T06:16:49,003 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - Submitting request to initiate the pipeline processing
2022-03-15T06:16:49,005 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [service-map-pipeline] - Initiating pipeline execution
2022-03-15T06:16:49,005 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [service-map-pipeline] - Submitting request to initiate the pipeline processing
2022-03-15T06:16:49,006 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [raw-pipeline] - Initiating pipeline execution
2022-03-15T06:16:49,006 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [raw-pipeline] - Submitting request to initiate the pipeline processing
2022-03-15T06:16:49,008 [main] INFO  com.amazon.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
2022-03-15T06:16:49,120 [entry-pipeline-prepper-worker-1-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  entry-pipeline Worker: No records received from buffer
2022-03-15T06:16:49,120 [service-map-pipeline-prepper-worker-3-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  service-map-pipeline Worker: No records received from buffer
2022-03-15T06:16:52,008 [raw-pipeline-prepper-worker-5-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  raw-pipeline Worker: No records received from buffer

But when I execut the curl command its failing to connect. Wat could be th reason

curl localhost:4900/list
curl: (7) Failed to connect to localhost port 4900 after 2236 ms: Connection refused

graytaylor0 · March 15, 2022, 3:40pm

Hi @ammujgd,

The reason the connection is refused is because SSL is enabled on the Data Prepper server in the Docker image by default. While there are ways to insecurely curl the endpoints, the easiest way to try out the core apis will be to create a data-prepper-config.yaml that contains ssl:false. Then override the default data-prepper-config.yaml by passing the created one to the Docker image

docker run \
 --name data-prepper-test \
 -p 21890:21890 -p 4900:4900 \
 -v ${PWD}/examples/config/example-pipelines.yaml:/usr/share/data-prepper/pipelines.yaml \
 -v ${PWD}/examples/config/example-data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml \
 opensearchproject/data-prepper:latest

ammujgd · March 15, 2022, 4:00pm

Thanks @graytaylor0 . I am actually starting data prepper by setting ssl to false in data-prepper-config.yaml. I am also starting data prepper as below like you suggested. Still it is failing

docker run --expose 21890 -v ‘/c/program files/docker/cli-plugins/pipeline.yaml:/usr/share/data-prepper/pipelines.yaml’ -v ‘/c/program files/docker/cli-plugins/data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml’ --network “host” opensearchproject/data-prepper:latest

graytaylor0 · March 15, 2022, 4:22pm

@ammujgd The issue is with your docker networking then. My suggestion would be to not use the “host” networking of Docker, and instead create your own bridge network that you attach to both the otel-collector and data-prepper containers. You will need to publish the necessary ports explicitly in order to curl from your local machine, and you will need to change the data prepper otel exporter to the data prepper docker dns/ip.

Also, here is an example of using a similar idea for Data Prepper log analytics. It may make it easier to see how the networking can be configured. (data-prepper/log_ingestion_demo_guide.md at main · opensearch-project/data-prepper · GitHub)

nateynate · March 15, 2022, 11:38pm

Hi @ammujgd -

I think the tell might be right here:

PS C:\Program Files\Docker\cli-plugins> docker run --expose 4900-v `/c/program files/docker/cli-plugins/pipeline.yaml:/usr/share/data-prepper/pipelines.yaml' -v `/c/program files/docker/cli-plugins/data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml' --network "host" opensearchproject/data-prepper:latest docker: invalid range format for --expose: 4900-v, error: strconv.ParseUint: parsing "v": invalid syntax. See 'docker run --help'.

Note the parsing "v": invalid syntax.

If I’m reading your invocation properly, it looks like you need to put a space after 4900 and before the next option (-v)

--expose 4900-v should become --expose 4900 -v

I bet that should at least get you more detailed information.

Nate

ammujgd · March 16, 2022, 6:20am

SO when I create my custom bridge, the open search can still run locally? Just the otel-collector and data prepper will be associated with the new network bridge?

The reason why I am asking is because my data prepper is starting only when network i s “host”. When i changed it to my custom network bridge , data prepper says connection refused with open search sink

docker run --name data-prepper2222222 -v `/c/program files/docker/cli-plugins/pipeline.yaml:/usr/share/data-prepper/pipelines.yaml' -v `/c/program files/docker/cli-plugins/data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml' --network "prepper-net" opensearchproject/data-prepper:latest
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2022-03-16T06:03:02,161 [main] INFO  com.amazon.dataprepper.pipeline.server.DataPrepperServer - Creating Data Prepper server without TLS
2022-03-16T06:03:02,181 [main] INFO        com.amazon.dataprepper.DataPrepper - Using /usr/share/data-prepper/pipelines.yaml configuration file
2022-03-16T06:03:02,231 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building pipeline [entry-pipeline] from provided configuration
2022-03-16T06:03:02,231 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [otel_trace_source] as source component for the pipeline [entry-pipeline]
2022-03-16T06:03:02,415 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [entry-pipeline]
2022-03-16T06:03:02,423 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building preppers for the pipeline [entry-pipeline]
2022-03-16T06:03:02,425 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [entry-pipeline]
2022-03-16T06:03:02,426 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as sink component
2022-03-16T06:03:02,427 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as sink component
2022-03-16T06:03:02,430 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building pipeline [service-map-pipeline] from provided configuration
2022-03-16T06:03:02,430 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as source component for the pipeline [service-map-pipeline]
2022-03-16T06:03:02,431 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [service-map-pipeline]
2022-03-16T06:03:02,433 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building preppers for the pipeline [service-map-pipeline]
2022-03-16T06:03:02,698 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [service-map-pipeline]
2022-03-16T06:03:02,698 [main] INFO  com.amazon.dataprepper.parser.PipelineParser - Building [opensearch] as sink component
2022-03-16T06:03:02,715 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Starting OpenSearch sink
2022-03-16T06:03:02,730 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2022-03-16T06:03:02,897 [main] INFO  com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2022-03-16T06:03:03,043 [main] ERROR com.amazon.dataprepper.plugins.PluginFactory - Encountered exception while instantiating the plugin OpenSearchSink
java.lang.reflect.InvocationTargetException: null
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
        at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) [?:?]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) [?:?]
        at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1]
Caused by: java.lang.RuntimeException: Connection refused
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:85) ~[data-prepper.jar:1.0.0.0-rc1]
        ... 19 more
Caused by: java.net.ConnectException: Connection refused
        at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:892) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestClient.performRequest(RestClient.java:296) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1]
        ... 19 more
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
        at sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:875) ~[?:?]
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[data-prepper.jar:1.0.0.0-rc1]
        at java.lang.Thread.run(Thread.java:832) ~[?:?]
2022-03-16T06:03:03,048 [main] ERROR com.amazon.dataprepper.parser.PipelineParser - Construction of pipeline components failed, skipping building of pipeline [service-map-pipeline] and its connected pipelines
com.amazon.dataprepper.plugins.PluginException: Encountered exception while instantiating the plugin OpenSearchSink
        at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:45) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.SinkFactory.newSink(SinkFactory.java:23) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:160) ~[data-prepper.jar:1.0.0.0-rc1]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
        at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:109) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:75) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:108) [data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:31) [data-prepper.jar:1.0.0.0-rc1]
Caused by: java.lang.reflect.InvocationTargetException
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
        at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1]
        ... 13 more
Caused by: java.lang.RuntimeException: Connection refused
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:85) ~[data-prepper.jar:1.0.0.0-rc1]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
        at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1]
        ... 13 more
Caused by: java.net.ConnectException: Connection refused
        at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:892) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestClient.performRequest(RestClient.java:296) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.IndexStateManagement.checkISMEnabled(IndexStateManagement.java:36) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.start(OpenSearchSink.java:92) ~[data-prepper.jar:1.0.0.0-rc1]
        at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:83) ~[data-prepper.jar:1.0.0.0-rc1]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
        at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:35) ~[data-prepper.jar:1.0.0.0-rc1]
        ... 13 more
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
        at sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:875) ~[?:?]
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[data-prepper.jar:1.0.0.0-rc1]
        at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[data-prepper.jar:1.0.0.0-rc1]
        at java.lang.Thread.run(Thread.java:832) ~[?:?]
2022-03-16T06:03:03,052 [main] ERROR       com.amazon.dataprepper.DataPrepper - No valid pipeline is available for execution, exiting

This is my new network bridge . Why does it not show any IP address ??

`
[
    {
        "Name": "prepper-net",
        "Id": "bd10b0346b94501da67e1c7f190db6d33eb304a9008639d978ae6a965845c80e",
        "Created": "2022-03-16T05:42:03.6900658Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]`

graytaylor0 · March 16, 2022, 3:34pm

@ammujgd You have a couple of options in regards to OpenSearch. One is to run OpenSearch through Docker and attach it to prepper-net. You can slightly modify the docker-compose.yaml from here.

The other option is to target port 9200 locally by changing your OpenSearch sink endpoints in your Data Prepper pipeline configuration. I believe it should work on Windows.

sink:
   - opensearch:
        hosts: ["https://host.docker.internal:9200"]

For consistency, I recommend the first approach (keeping everything in Docker)

ammujgd · March 16, 2022, 4:37pm

Thanks

Still I don’t see the data comin in to the prepper…Is the below config right?

I can see collector s getting the data bt the exporter is not sending it to dataprepper. I have used the dataprepper docker container Ip address as you suggested. What could be the reason for no stream at prepper

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: host.docker.internal:4317
      http:
        endpoint: host.docker.internal:55681

processors:
  batch:

exporters:
  otlp/data-prepper:
    endpoint: 172.20.0.3:21890
    insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/data-prepper]

My docker network inspect

[
    {
        "Name": "prepper-net",
        "Id": "bd10b0346b94501da67e1c7f190db6d33eb304a9008639d978ae6a965845c80e",
        "Created": "2022-03-16T05:42:03.6900658Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "5e6184cead4a9db44a362f3f5788ebe65e77de9dd2303b5724a4e7261de0f4a7": {
                "Name": "otelcol99",
                "EndpointID": "7a5960a7df9a55667dcf53ed01dcb1874e860c90d47a785bd0343b5dee277d08",
                "MacAddress": "02:42:ac:14:00:03",
                "IPv4Address": "172.20.0.3/16",
                "IPv6Address": ""
            },
            "5e6233f19e6454c1eef80124635dbce1c0a952adcd695c40dc081a24b12ce39a": {
                "Name": "sad_mcclintock",
                "EndpointID": "eba4765046592913c757acf51fdfd4c36eca258b3b658f7dde499c740609e419",
                "MacAddress": "02:42:ac:14:00:02",
                "IPv4Address": "172.20.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

graytaylor0 · March 17, 2022, 5:14pm

@ammujgd I don’t understand which container is which in your docker network inspect output. Is otelcol99 the otel collector or Data Prepper? Because it has an IP address of 172.20.0.03, which is what you set as the Data Prepper endpoint.

ammujgd · March 17, 2022, 5:43pm

Hi @graytaylor0

otelcol99 is the otelcollector and sad_mcclintocks is the data prepper.

This “IPv4Address”: “172.20.0.2/16”, indicates the Ip range ?. SO what exactly should I be using the Ip address in exporter?
I tried with 172.20.0.2 , still no data recieved in dataprepper.

graytaylor0 · March 17, 2022, 7:19pm

@ammujgd What is the output of

docker inspect 5e6233f19e6454c1eef80124635dbce1c0a952adcd695c40dc081a24b12ce39a | grep "IPAddress"`

ammujgd · March 18, 2022, 2:52am

This is the result

 "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "",

graytaylor0 · March 18, 2022, 4:44pm

@ammujgd Interesting that there is no IpAddress assigned. The good news is that when you create a user-defined bridge network, the name assigned to a container is resolvable by dns with docker. So if you run data prepper with docker run --name sad_mcclintock --network prepper-net... then you should be able to add the name of the container to your otel-collector-config like this

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: host.docker.internal:4317
      http:
        endpoint: host.docker.internal:55681

processors:
  batch:

exporters:
  otlp/2:
    endpoint: sad_mcclintock:21890
    insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/2]

ammujgd · March 18, 2022, 5:40pm

Thanks @graytaylor0

I tried starting data prepper like you suggested , but the container name is already in use

docker: Error response from daemon: Conflict. The container name "/sad_mcclintock" is already in use by container "5e6233f19e6454c1eef80124635dbce1c0a952adcd695c40dc081a24b12ce39a". You have to remove (or rename) that container to be able to reuse that name.

Seems like it didnt like that name

graytaylor0 · March 18, 2022, 6:06pm

@ammujgd You have to delete the old container before creating a new one of the same name. Are you using Docker Desktop? It is rather easy to delete containers through Docker Desktop. You should also be able to delete the container by running

docker rm /sad_mcclintock

ammujgd · March 18, 2022, 6:22pm

Still no luck …Data prepper says

2022-03-18T18:20:56,339 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [service-map-pipeline] - Submitting request to initiate the pipeline processing
2022-03-18T18:20:56,340 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [raw-pipeline] - Initiating pipeline execution
2022-03-18T18:20:56,341 [main] INFO  com.amazon.dataprepper.pipeline.Pipeline - Pipeline [raw-pipeline] - Submitting request to initiate the pipeline processing
2022-03-18T18:20:56,342 [main] INFO  com.amazon.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
2022-03-18T18:20:56,445 [entry-pipeline-prepper-worker-1-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  entry-pipeline Worker: No records received from buffer
2022-03-18T18:20:56,445 [service-map-pipeline-prepper-worker-3-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  service-map-pipeline Worker: No records received from buffer
2022-03-18T18:20:59,343 [raw-pipeline-prepper-worker-5-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  raw-pipeline Worker: No records received from buffer

graytaylor0 · March 21, 2022, 2:57pm

@ammujgd Do you have an otel-collector logs you can show? You can use the logging exporter (opentelemetry-collector/README.md at main · open-telemetry/opentelemetry-collector · GitHub)

Topic		Replies	Views
Data Prepper not starting Data Prepper	24	4515	November 11, 2021
Data-preeper plugin Data Prepper troubleshoot	8	1024	January 24, 2022
Latest data-prepper not working Data Prepper	10	2192	September 28, 2022
Can't configure Data Prepper with Opensearch Data Prepper troubleshoot , configure	2	1863	February 11, 2024
Error whle starting data prepper org.opensearch.OpenSearchStatusException: Unable to parse response body Data Prepper troubleshoot	2	1381	March 14, 2022

My data prepper has started but why is this curl command failing

Related topics