Versions OpenSearch 2.10/Dashboard 2.10/Server OS MacOS/Browser Docker
Describe the issue:
I setup a docker-compose file to try to send Node.js traces to Opensearch using Opentelemetry and Data prepper but I keep getting error messages from opentelemetry that it is not able to connect to data prepper.
Data prepper is also displaying an error about not being able to create the opensearch sink connector Caused by: java.io.IOException: opensearch: Name or service not known
I tried changing the name of the service in the data-prepper config but that did not work.
Configuration:
Here are the docker-compose files for Opentelemetry, Data Prepper, Opensearch and Opensearch-Dashboards
version: "3"
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:0.59.0
command:
- "--config"
- "/otel-local-config.yaml"
volumes:
- ./tracetest/collector.config.yaml:/otel-local-config.yaml
depends_on:
data-prepper:
condition: service_started
data-prepper:
restart: unless-stopped
image: opensearchproject/data-prepper:1.5.1
volumes:
- ./tracetest/opensearch/opensearch-analytics.yaml:/usr/share/data-prepper/pipelines.yaml
- ./tracetest/opensearch/opensearch-data-prepper-config.yaml:/usr/share/data-prepper/data-prepper-config.yaml
depends_on:
opensearch:
condition: service_started
opensearch:
# container_name: node-3.example.com
image: opensearchproject/opensearch:2.10.0
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
ports:
- "9200:9200"
- "9600:9600"
# healthcheck:
# test: curl -s http://localhost:9200 >/dev/null || exit 1
# interval: 5s
# timeout: 10s
# retries: 5
networks:
- my_network
dashboards:
image: opensearchproject/opensearch-dashboards:2.10.0
container_name: opensearch-dashboards3
ports:
- "5601:5601"
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch:9200"]'
depends_on:
- opensearch
networks:
- my_network
networks:
my_network:
The node.js app is in a separate docker-compose.yml file below:
version: '3'
services:
app:
image: quick-start-nodejs
build: .
ports:
- "8080:8080"
networks:
- my_network
networks:
my_network:
this is the Dockerfile also for the node.js app:
FROM node:slim
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "npm", "run", "with-grpc-tracer" ]
I used this document [Running Tracetest with OpenSearch | Tracetest Docs](Running Tracetest with OpenSearch | Tracetest Docs to get things up and running but then the configuration did not have opensearch-dashboards configured so I tried to add opensearch-dashboards myself but authorization and authentication was disabled in this configuration so I was unable to login in the web admin interface of Opensearch. In the end I decided to make my own docker-compose from scratch which led to the above errors.
Relevant Logs or Screenshots:
data prepper docker logs
2023-10-05 13:55:29 WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2023-10-05 13:55:50 2023-10-05T11:55:50,660 [main] INFO com.amazon.dataprepper.parser.config.DataPrepperAppConfiguration - Command line args: /usr/share/data-prepper/pipelines.yaml,/usr/share/data-prepper/data-prepper-config.yaml
2023-10-05 13:55:50 2023-10-05T11:55:50,695 [main] INFO com.amazon.dataprepper.parser.config.DataPrepperArgs - Using /usr/share/data-prepper/pipelines.yaml configuration file
2023-10-05 13:56:32 2023-10-05T11:56:32,488 [main] WARN com.amazon.dataprepper.parser.model.PipelineConfiguration - Prepper configurations are deprecated, processor configurations will be required in Data Prepper 2.0
2023-10-05 13:56:32 2023-10-05T11:56:32,559 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building pipeline [entry-pipeline] from provided configuration
2023-10-05 13:56:32 2023-10-05T11:56:32,570 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building [otel_trace_source] as source component for the pipeline [entry-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,168 [main] WARN com.amazon.dataprepper.plugins.source.oteltrace.OTelTraceSource - Creating otel-trace-source without authentication. This is not secure.
2023-10-05 13:56:33 2023-10-05T11:56:33,170 [main] WARN com.amazon.dataprepper.plugins.source.oteltrace.OTelTraceSource - In order to set up Http Basic authentication for the otel-trace-source, go here: https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/otel-trace-source#authentication-configurations
2023-10-05 13:56:33 2023-10-05T11:56:33,187 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [entry-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,491 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building processors for the pipeline [entry-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,500 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [entry-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,506 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as sink component
2023-10-05 13:56:33 2023-10-05T11:56:33,528 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building pipeline [raw-pipeline] from provided configuration
2023-10-05 13:56:33 2023-10-05T11:56:33,531 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building [pipeline] as source component for the pipeline [raw-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,533 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building buffer for the pipeline [raw-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,564 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building processors for the pipeline [raw-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,573 [main] WARN com.amazon.dataprepper.parser.PipelineParser - No plugin of type Processor found for plugin setting: otel_trace_raw_prepper, attempting to find comparable Prepper plugin.
2023-10-05 13:56:33 2023-10-05T11:56:33,745 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building sinks for the pipeline [raw-pipeline]
2023-10-05 13:56:33 2023-10-05T11:56:33,751 [main] INFO com.amazon.dataprepper.parser.PipelineParser - Building [opensearch] as sink component
2023-10-05 13:56:34 2023-10-05T11:56:34,109 [main] INFO com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink - Initializing OpenSearch sink
2023-10-05 13:56:36 2023-10-05T11:56:36,857 [main] INFO com.amazon.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2023-10-05 13:56:38 2023-10-05T11:56:38,087 [main] ERROR com.amazon.dataprepper.plugin.PluginCreator - Encountered exception while instantiating the plugin OpenSearchSink
2023-10-05 13:56:38 java.lang.reflect.InvocationTargetException: null
2023-10-05 13:56:38 at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
2023-10-05 13:56:38 at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
2023-10-05 13:56:38 at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
2023-10-05 13:56:38 at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
2023-10-05 13:56:38 at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:40) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugin.DefaultPluginFactory.loadPlugin(DefaultPluginFactory.java:66) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:180) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
2023-10-05 13:56:38 at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
2023-10-05 13:56:38 at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
2023-10-05 13:56:38 at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
2023-10-05 13:56:38 at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
2023-10-05 13:56:38 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
2023-10-05 13:56:38 at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
2023-10-05 13:56:38 at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:107) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:72) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.DataPrepper.<init>(DataPrepper.java:57) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
2023-10-05 13:56:38 at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
2023-10-05 13:56:38 at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
2023-10-05 13:56:38 at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
2023-10-05 13:56:38 at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
2023-10-05 13:56:38 at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:541) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:541) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.ContextManager.<init>(ContextManager.java:48) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:22) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 Caused by: java.lang.RuntimeException: opensearch: Name or service not known
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:92) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 ... 82 more
2023-10-05 13:56:38 Caused by: java.io.IOException: opensearch: Name or service not known
2023-10-05 13:56:38 at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:912) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.RestClient.performRequest(RestClient.java:301) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.RestClient.performRequest(RestClient.java:289) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1762) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1728) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:119) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.index.IndexManager.checkISMEnabled(IndexManager.java:149) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.index.IndexManager.checkAndCreateIndexTemplate(IndexManager.java:165) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.index.IndexManager.setupIndex(IndexManager.java:160) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.initialize(OpenSearchSink.java:105) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:89) ~[data-prepper.jar:1.5.1]
2023-10-05 13:56:38 ... 82 more
opentelemetry docker logs:
2023-10-05 13:55:25 2023/10/05 11:55:25 proto: duplicate proto type registered: jaeger.api_v2.PostSpansRequest
2023-10-05 13:55:25 2023/10/05 11:55:25 proto: duplicate proto type registered: jaeger.api_v2.PostSpansResponse
2023-10-05 13:55:25 2023-10-05T11:55:25.683Zinfoservice/telemetry.go:115Setting up own telemetry...
2023-10-05 13:55:25 2023-10-05T11:55:25.685Zinfoservice/telemetry.go:156Serving Prometheus metrics{"address": ":8888", "level": "basic"}
2023-10-05 13:55:25 2023-10-05T11:55:25.692Zinfoservice/service.go:112Starting otelcol-contrib...{"Version": "0.59.0", "NumCPU": 4}
2023-10-05 13:55:25 2023-10-05T11:55:25.693Zinfoextensions/extensions.go:42Starting extensions...
2023-10-05 13:55:25 2023-10-05T11:55:25.696Zinfopipelines/pipelines.go:74Starting exporters...
2023-10-05 13:55:25 2023-10-05T11:55:25.697Zinfopipelines/pipelines.go:78Exporter is starting...{"kind": "exporter", "data_type": "traces", "name": "otlp/2"}
2023-10-05 13:55:25 2023-10-05T11:55:25.708Zinfopipelines/pipelines.go:82Exporter started.{"kind": "exporter", "data_type": "traces", "name": "otlp/2"}
2023-10-05 13:55:25 2023-10-05T11:55:25.711Zinfopipelines/pipelines.go:86Starting processors...
2023-10-05 13:55:25 2023-10-05T11:55:25.713Zinfopipelines/pipelines.go:90Processor is starting...{"kind": "processor", "name": "batch", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.713Zinfopipelines/pipelines.go:94Processor started.{"kind": "processor", "name": "batch", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.713Zinfopipelines/pipelines.go:98Starting receivers...
2023-10-05 13:55:25 2023-10-05T11:55:25.713Zinfopipelines/pipelines.go:102Receiver is starting...{"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.714Zinfootlpreceiver/otlp.go:70Starting GRPC server on endpoint 0.0.0.0:4317{"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.715Zinfootlpreceiver/otlp.go:88Starting HTTP server on endpoint 0.0.0.0:4318{"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.715Zinfopipelines/pipelines.go:106Receiver started.{"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2023-10-05 13:55:25 2023-10-05T11:55:25.715Zinfoservice/service.go:129Everything is ready. Begin running and processing data.
2023-10-05 13:55:25 2023-10-05T11:55:25.759Zwarnzapgrpc/zapgrpc.go:191[core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {
2023-10-05 13:55:25 "Addr": "data-prepper:21890",
2023-10-05 13:55:25 "ServerName": "data-prepper:21890",
2023-10-05 13:55:25 "Attributes": null,
2023-10-05 13:55:25 "BalancerAttributes": null,
2023-10-05 13:55:25 "Type": 0,
2023-10-05 13:55:25 "Metadata": null
2023-10-05 13:55:25 }. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.32.2:21890: connect: connection refused"{"grpc_log": true}