@pc_9393 I reproduced this exact setup locally: Filebeat 8.18.1 → Logstash 9.4.0 → OpenSearch 2.19.1, see following details:
override_main_response_version not needed
With logstash-output-opensearch (the dedicated OpenSearch plugin, not the generic Elasticsearch one), this setting is unnecessary. The plugin correctly detects OpenSearch natively. You can see it in the Logstash startup logs:
Cluster version determined (2.19.1) {version: 2}
No version confusion and no workaround required.
ecs_compatibility
Out of the box with Logstash 9.x, the pipeline defaults to ecs_compatibility: v8. This causes logstash-output-opensearch 2.1.1 to attempt installing an ECS v8 index template into OpenSearch, which immediately throws:
ERROR Failed to install template - caused by: #<NoMethodError: undefined method 'exists?' for class File>
The root cause is that File.exists? was removed in Ruby 3.x, and Logstash 9.x ships with JRuby 10 (Ruby 3.4). The plugin has a bug in its template_manager.rb that hits this on every startup regardless of ECS mode.
The fix is two-part:
- In
logstash.yml: disable ECS at the pipeline level:
pipeline.ecs_compatibility: disabled
Note, this must go in logstash.yml, not in logstash.conf. The opensearch output plugin does not expose ecs_compatibility as a plugin-level config option, so putting it inside the output block will cause a parse error.
- In
logstash.conf, skip template management entirely:
output {
opensearch {
hosts => ["https://your-opensearch:9200"]
index => "logstash-%{+YYYY.MM.dd}"
user => "admin"
password => "your-password"
ssl => true
ssl_certificate_verification => true
cacert => "/path/to/root-ca.pem"
manage_template => false # avoids the File.exists? bug in plugin 2.1.1
}
}
Hope this helps