docker compose file for dataprepper
version: ‘3’
services:
data-prepper:
container_name: data-prepper
image: opensearchproject/data-prepper
volumes:
- ./log_pipeline.yaml:/usr/share/data-prepper/pipelines/log_pipeline.yaml
ports:
- 2021:2021
networks:
- opensearch-net
networks:
opensearch-net:
[here i had mentioned depends opon as well, was getting error for service name]
opensearch and opensearch dashboard are running properly
Adding log_pipeline.yaml
log-pipeline:
source:
http:
ssl: false
processor:
- grok:
match:
log: [ “%{COMMONAPACHELOG}” ]
sink:
- opensearch:
hosts: [ “http://opensearch-node1:9200”, “http://opensearch-node2:9200” ]
insecure: true
username: admin
password: admin
index: nginx_logs
getting below error
org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink, retrying: opensearch-node2