Is there a simple way to ingest cloudwatch logs to ElasticSearch?
Or is the lambda way the only easy way?
The setup I have done in the past for this is to create a Kinesis Data Stream (lets call it logstash-kinesis)
From there, assuming you are asking about indexing Cloudwatch log groups into elasticsearch, for each log group you have, you want to add a subscription filter to it, pointing that log group’s subscription filter to the kinesis data stream (logstash-kinesis)…
From there you can run an indexing process like logstash using the cloudwatch codec and input directly from the kinesis data stream (logstash-kinesis)
ie:
input {
kinesis {
kinesis_stream_name => "logstash-kinesis"
application_name => "logstash-kinesis"
region => "us-west-2"
codec => cloudwatch_logs
}
}
...