Logstash Auth with IAM

Security
1

Hello all,
in our company, we are using the amazon elasticsearch service. Not so long ago, we switched authentication of elasticsearch to cognito and webSSO. Well we have now found a problem with the logstash. Logstash previously used basic authentication to transfer messages to ES.

Following steps were taken from us to solve the problem:

  • we have installed amazon_es plugin for logstash
  • we have created iam policy to allow access to elasticsearch
  • we have created role and have attached the previously created policy to it and we have set EC2 as a trusted entity

So instead of creating a access key and secret key we want using iam role.

Nonetheless, logstash gets 403 upon login. Can anyone tell me what I forgot?

IAM Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "es:ESHttpHead",
                "es:ESHttpPost",
                "es:ESHttpGet",
                "es:ESHttpPut"
            ],
            "Resource": "arn:aws:es:eu-central-1:xxxxxxxxxx:domain/my-es-domain/*"
        }
    ]
}

ElasticSearch Access Policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Enable open access",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::xxxxxxxxxx:role/logstash-system-es",
          "*"
        ]
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:eu-central-1:xxxxxxx:domain/my-es-domain/*"
    }
  ]
}

Finally logstash config:

output{
	amazon_es {
      hosts => ["es-domain"]
      ssl => true
	  region => "eu-central-1"
	  aws_access_key_id => ''
      aws_secret_access_key => ''
	  index   => "%{[@metadata][es_index]}"
    }
	stdout {
        codec => rubydebug {metadata => true}
    }
}
2

I have found the solution to my problem.
The iam role must also be added as backend role in elasticsearch!