Filebeat & Logstash version upgrade compatibility with AWS OpenSearch 2.19 on EKS — what is the correct stable version?

OpenSearch
1

OpenSearch AWS OpenSearch Service v2.19
Filebeat (current)8.19.7
(OSS)Logstash (current)8.19.7- Deployed as Containers on Amazon EKS
Workflow: Filebeat → Logstash → OpenSearch

We are running a logging pipeline on EKS where Filebeat ships to Logstash, and Logstash writes to AWS OpenSearch 2.19 via the logstash-output-opensearch plugin. We want to upgrade both components to the latest stable version and have a few questions:

What is the latest stable Logstash OSS version confirmed to work with AWS OpenSearch 2.19 and the logstash-output-opensearch plugin?

  1. Is Logstash 9.x (e.g. 9.4.1) supported with OpenSearch 2.19? The official COMPATIBILITY.md only covers up to 7.13.2, and AWS docs mention 8.1 and lower. Has anyone tested 9.x against OpenSearch 2.x?

  2. Since Filebeat only talks to Logstash (not OpenSearch directly), is there any version constraint on Filebeat 8.18.x or 9.x as a forwarder?

We have reviewed the official compatibility matrix and plugin COMPATIBILITY.md but found no entry for Logstash 9.x against OpenSearch 2.x. Any guidance on a confirmed, production-safe version combination — including any required config changes like ecs_compatibility or override_main_response_version — would be greatly appreciated. Thank you.

2

@pc_9393 I reproduced this exact setup locally: Filebeat 8.18.1 → Logstash 9.4.0 → OpenSearch 2.19.1, see following details:

override_main_response_version not needed

With logstash-output-opensearch (the dedicated OpenSearch plugin, not the generic Elasticsearch one), this setting is unnecessary. The plugin correctly detects OpenSearch natively. You can see it in the Logstash startup logs:

Cluster version determined (2.19.1) {version: 2}

No version confusion and no workaround required.

ecs_compatibility

Out of the box with Logstash 9.x, the pipeline defaults to ecs_compatibility: v8. This causes logstash-output-opensearch 2.1.1 to attempt installing an ECS v8 index template into OpenSearch, which immediately throws:

ERROR Failed to install template - caused by: #<NoMethodError: undefined method 'exists?' for class File>

The root cause is that File.exists? was removed in Ruby 3.x, and Logstash 9.x ships with JRuby 10 (Ruby 3.4). The plugin has a bug in its template_manager.rb that hits this on every startup regardless of ECS mode.

The fix is two-part:

  1. In logstash.yml: disable ECS at the pipeline level:
pipeline.ecs_compatibility: disabled

Note, this must go in logstash.yml, not in logstash.conf. The opensearch output plugin does not expose ecs_compatibility as a plugin-level config option, so putting it inside the output block will cause a parse error.

  1. In logstash.conf, skip template management entirely:
output {
  opensearch {
    hosts => ["https://your-opensearch:9200"]
    index => "logstash-%{+YYYY.MM.dd}"
    user => "admin"
    password => "your-password"
    ssl => true
    ssl_certificate_verification => true
    cacert => "/path/to/root-ca.pem"
    manage_template => false   # avoids the File.exists? bug in plugin 2.1.1
  }
}

Hope this helps