Logstash opensearch filter plugin

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.11.1

Describe the issue:

opensearch filter plugin for logstash is not working.

Configuration:

Below logstash config gives the error-
[ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<Faraday::SSLError wrapped=#<OpenSSL::SSL::SSLError: certificate verify failed>>

input { ... }

filter {
	opensearch {
		hosts => ["https://localhost:9200"]
		# ssl => true
		# ssl_certificate_verification => false
		index => "students"
		user => "admin"
		password => "admin"
		query_template => "es-query/student_id_query.json"
		docinfo_fields => {
			"_id" => "student_id"
		}
	}
}

output { ... }

Then if I change the filter plugin like this (using ssl_certificate_verification => false), the below error occurs-

[ERROR][logstash.filters.opensearch] Unknown setting 'ssl_certificate_verification' for opensearch

input { ... }

filter {
	opensearch {
		hosts => ["https://localhost:9200"]
		#ssl => true
		ssl_certificate_verification => false
		index => "students"
		user => "admin"
		password => "admin"
		query_template => "es-query/student_id_query.json"
		docinfo_fields => {
			"_id" => "student_id"
		}
	}
}

output { ... }

If I remove https:// from the host and enable ssl => true, then also same error occurs.

Please look into the filter plugin for logstash-

Relevant Logs or Screenshots:

[ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>“main”, :exception=>#<Faraday::SSLError wrapped=#<OpenSSL::SSL::SSLError: certificate verify failed>>

[ERROR][logstash.filters.opensearch] Unknown setting ‘ssl_certificate_verification’ for opensearch

2

I think the correct way to disable certificate verification is

ssl_verification_mode: none

Then the correct config would be

filter {
	opensearch {
		hosts => ["https://localhost:9200"]
		ssl => true
		ssl_verification_mode => none
		index => "students"
		user => "admin"
		password => "admin"
		query_template => "es-query/student_id_query.json"
		docinfo_fields => {
			"_id" => "student_id"
		}
	}
}
3

Hi @pjanzen,

Thank you for replying.

As I mentioned before, the setting ssl_verification_mode is unknown to opensearch filter plugin here. Although it works in the output plugin.

With the suggested configuration, I received the below error-

[2024-02-06T21:31:00,890][ERROR][logstash.filters.opensearch] Unknown setting 'ssl_certificate_verification' for opensearch
[2024-02-06T21:31:00,894][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration."
4

I have also provided the details in the issue-