Beats OSS with OpenSearch 3.x

Hi,

I would like to use Beats with OpenSearch 3.1. I found the compatibility matrices at Tools - OpenSearch Documentation. However, it only contains a row for OpenSearch 1.x to 2.x (no OpenSearch 3.x). Are the Beats OSS 7.12.x* versions compatible with OpenSearch 3.x?

I am trying to avoid using Logstash as a middleware between Beats and OpenSearch, because I need to authenticate the individual VMs running Beats to collect data about the commands issued on them. I.e. each VM should have a set of credentials and write access to its own index. No other VM can write to that index.

Thanks,

Marek

To extend, I’ve tried running OpenSearch 3.1 together with Filebeat OSS 7.12.1, but was unable to change the necessary override_main_response_version setting:

{"error":{"root_cause":[{"type":"settings_exception","reason":"persistent setting [compatibility.override_main_response_version], not recognized"}],"type":"settings_exception","reason":"persistent setting [compatibility.override_main_response_version], not recognized"},"status":400}

Was this option removed in OpenSearch 3.0/3.1? I don’t see such change in the changelog.

Elastic has introduced a version check in Filebeat 7.13. It won’t allow you to connect with OpenSearch.

As a result compatibility.override_main_response_version has been removed in 3.X.

You can try using Logstash with the OpenSearch output plugin and newer Filebeat as a workaround.

2 Likes