Use Elasticsearch’s _rollover API For Efficient Storage Distribution | Open Distro

General Feedback
1

Many Open Distro for Elasticsearch users manage data life cycle in their clusters by creating an index based on a standard time period, usually one index per day. This pattern has many advantages: ingest tools like Logstash support index rollover out of the box; defining a retention window is straightforward; and deleting old data is as simple as dropping an index.

This is a companion discussion topic for the original entry at https://opendistro.github.io/for-elasticsearch/blog/open%20distro%20for%20elasticsearch%20updates/2019/08/Use-Elasticsearchs-_rollover-API-For-Efficient-Storage-Distribution/
2

How would one implement this when all log ingestion happens in logstash? We receive all logs first at logstash and then logstash will send it to Elasticsearch. We want to utilize the rollover API but we need logstash to send the logs it recevies to a elasticsearch alias if im correct?

3

@victor If you look at the docs it creates an alias that points at the numbered index, i think you would just ship your logs to the alias and then when the limit is hit the data flows to the next index in the sequence.

1 Like