Data Prepper - HTTP/1.1 403 Forbidden

Data Prepper
1

Hello,

I trying to run the data prepper on kubernetes.

I have the following errors :

Caused by: org.elasticsearch.ElasticsearchStatusException: method [HEAD], host [https://elasticsearch.*.pl:9200], URI [/_alias/otel-v1-apm-span?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=true], status line [HTTP/1.1 403 Forbidden]

[main] ERROR com.amazon.dataprepper.parser.PipelineParser - Construction of pipeline components failed, skipping building of pipeline [sample-pipeline] and its connected pipelines
com.amazon.dataprepper.plugins.PluginException: Encountered exception while instantiating the plugin ElasticsearchSink

The data prepper got all the permissions it was calling for :slight_smile:

Pipeline :
sample-pipeline:
workers: 4 # the number of workers
delay: 100 # in milliseconds, how long workers wait between read attempts
source:
otel_trace_source:
ssl: false
buffer:
bounded_blocking:
buffer_size: 1024 # max number of records the buffer accepts
batch_size: 256 # max number of records the buffer drains after each read
prepper:
- otel_trace_raw_prepper:
sink:
- elasticsearch:
hosts: [“https://elasticsearch.*.pl:9200”]
cert: “/usr/share/data-prepper/root-ca.pem”
username: “data-prepper”
password: “*****”
trace_analytics_raw: true

It simple configuration - I wish it would just start work :smiley:

THX
Maciej

2

Hello Maciej,

I’m sorry you are having trouble running Data Prepper. It appears that you are getting the 403 from OpenSearch/ElasticSearch itself. Data Prepper is trying to check if a specific alias exists, but doesn’t have permission to make this request.

This security page outlines what permissions are needed for use in Amazon OpenSearch Service.

The same permissions should apply for OpenSearch however it is run or managed. Looking at the bottom section, I’d first like to check that your data-prepper user has the all_access role or equivalent.

David

1 Like