OpenSearch logstash 7.16.2 issue with AWS elasticsearch input

OpenDistro
1

I’m using AWS Elasticsearch 7.10 service as server.
When using logstash with elsaticsearch input I get an issue of incompatible server version:

09:01:49.044 [[main]-pipeline-manager] ERROR logstash.javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Could not connect to a compatible version of Elasticsearch>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.12.1/lib/logstash/inputs/elasticsearch.rb:481:in `test_connection!'", "/opt/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.12.1/lib/logstash/inputs/elasticsearch.rb:243:in `register'", "/opt/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:232:in `block in register_plugins'", "org/jruby/RubyArray.java:1821:in `each'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:231:in `register_plugins'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:390:in `start_inputs'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:315:in `start_workers'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:189:in `run'", "/opt/logstash/logstash-core/lib/logstash/java_pipeline.rb:141:in `block in start'"], "pipeline.sources"=>["/etc/logstash/logstash-teams.conf"], :thread=>"#<Thread:0x6eab5ba7 run>"}

The pipeline input:

input {
	elasticsearch {
		hosts => ["${ELASTICSEARCH_ADDRESS}"]
		index => "logs-*"
		query => '{
			"query": {
				"bool": {
					"must": [
						{"range": {"@timestamp": {"gte": "now-60m", "lte": "now"}}}
					]
				}
			}
		}'
		size => 500
		scroll => "2m"
		docinfo => true
		schedule => "* * * * *"
	}
}
...

I have no issue with following versions:
Elasticsearch logstash-oss-7.12.0
Opensearch logstash 7.13.2

I got the issue with versions:
Elasticsearch logstash-oss-7.16.1
Opensearch logstash 7.16.2

(I didn’t test other version between those versions)

Opensearch output plugin fixes same kind of issue, I think the same process should be needed for input.

2

You should use the same logstash version as your AWS ES version, Elasticsearch changed their license
because of some issues related to the license of Elasticsearch. I remember reading it somewhere in Github :laughing:

3

I need to upgrade logstash to an up to date version due to log4j security issues

4

There is a workaround for not upgrading.

The widespread flag -Dlog4j2.formatMsgNoLookups=true is NOT sufficient to mitigate the vulnerability in Logstash in all cases, as Logstash uses Log4j in a way where the flag has no effect. If the user cannot upgrade to Logstash 7.16.2 or 6.8.22, it is necessary to remove the JndiLookup class from the log4j2 core jar, with the following command (which is applicable for 5.x, 6.x, and 7.x):

zip -q -d <LOGSTASH_HOME>/logstash-core/**/*/log4j-core-2.* org/apache/logging/log4j/core/lookup/JndiLookup.class

Please note that a restart of the Logstash process is necessary for the change to take effect.

Reference: Logstash update log4j version - Logstash - Discuss the Elastic Stack

Btw you may be considering switching to OpenSearch version to get the latest update ( I know it will take time but it should need to be done )

5

you could try using amazon_es or opensearch output plugins for logstash and it should work out of box.
elastic updated default elasticsearch output plugin to not work with non licensed elasticsearch…

6

Yes output plugin fixed exactly that issue, for output.
Here I’m having same issue but with input, that’s why I think an input plugin is needed based on the same model.