Data Prepper Distributions Update

Data Prepper 1.0 was released as two distributions - OpenSearch Data Prepper and another for Open Distro Data Prepper. Starting in Data Prepper 1.1, there is only one distribution, the OpenSearch Data Prepper. All active development is happening in the OpenSearch Data Prepper distribution. The last release of the Open Distro Data Prepper distribution was 1.0 in May 2021. New versions will only be supplied for critical security issues through Dec 2021.

If you haven’t yet migrated to the OpenSearch distribution, I encourage you to do so. Data Prepper 1.1 has new features and improvements, and there are even more planned for Data Prepper 1.2. Check out the roadmap to find out what is upcoming.

The OpenSearch distribution supports OpenSearch, Open Distro, and ElasticSearch 7.x as a sink. The migration is straightforward.

First, The elasticsearch sink has changed to opensearch. You will need to change your existing pipeline to use the opensearch plugin instead of elasticsearch.

Second, you need to to change your Data Prepper Docker image. The Open Distro Data Prepper Docker image was located at amazon/opendistro-for-elasticsearch-data-prepper. You will need to change this value to opensearchproject/opensearch-data-prepper.