I setup AWS Elasticsearch with Cognito as the IDP and i can log into Kibana without any issues. However, when I use logstash elasticsearch output using a user from the pool, I get
[2020-07-30T18:22:13,586][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error.
{:url=>"https://someuser:xxxxxx@blah-blah-host.someregion.es.amazonaws.com:443/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'https://blah-blah-host.someregion.es.amazonaws.com:443/'"}
I also cannot use basic auth. It returns
401 Authentication finally failed
How can i use credentials from Cognito in logstash to authenticate and output to elastic search?
Or, perhaps, is there another way to accomplish this?
My config:
input {
file {
path => "/path/to/log/*.log"
start_position => "beginning"
sincedb_path => "/path/to/sincedb"
}
}
output {
elasticsearch {
hosts => ["https://blah-blah-host.someregion.es.amazonaws.com:443"]
user => "someuser"
password => "S0m3P@$$w0rd"
index => "very-important-logs"
}
}