after reading countless documentations to AWS and Data Prepper for a couple of days I’ve come to the conclusion that I don’t understand how the authentication to the SQS Queue for log ingestion out of S3 works.
As far as I understand, I have to create an IAM Role that has access to the SQS queue that I created, and I call that queue via the “sts_role_arn” parameter in the pipeline.yaml
But the IAM Role needs a principal that dictates who can assume this role. So somehow Data Prepper has to authenticate to the IAM Role if I’m correct.
I found this GitHub issue where an access_key_id and a secret_key_id are mentioned, which at least could be used to authenticate to an IAM Account, but it seems that these parameters don’t exist anymore or never existed in the first place?
I hope someone can tell me what I’m missing here