I have the latest Opensearch/Dataprepper (2.18).
This pipeline works fine without this directives
template_type: index-template
template_content: >
…
sink:
- opensearch:
hosts: [ https://...... ]
insecure: true
username: "...."
password: "....."
index: syslog-standard
dlq_file: "/usr/share/data-prepper/pipelines/dlq_syslog.err"
template_type: index-template
template_content: >
{
"template" : {
"mappings" : {
"properties": {
"logsource": { "type" : "text" },
"program": { "type" : "text" },
"pid": { "type" : "text" },
"timestampOriginalString": { "type" : "text" },
"message": { "type" : "wildcard" }
}
}
}
}
But with them included I got errors in dataprepper logs:
2025-02-03T13:20:28,398 [syslog-pipeline-sink-worker-4-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink, retrying: Forbidden access
2025-02-03T13:20:28,602 [syslog-pipeline-sink-worker-4-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [syslog-pipeline] - sink is not ready for execution, retrying
2025-02-03T13:20:28,603 [syslog-pipeline-sink-worker-4-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Initializing OpenSearch sink
2025-02-03T13:20:28,603 [syslog-pipeline-sink-worker-4-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2025-02-03T13:20:28,603 [syslog-pipeline-sink-worker-4-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
Why?
Update: because of user permissions… The user I used (not admin) had enough rights to create index, but not for template creation. When I used admin user, it started to work.
P.