Setting up filebeat module ibmmq on opensearch

OpenSearch
1

opensearch 2.10.0, filebeat 7.17.13 logstash 8.9.0 (relevant - OpenSearch/Dashboard/Server OS/Browser):

I have a setup on my local machine of filebeat → logstash → opensearch trying to mirror production which has filebeat → logstash → elasticsearch as poc for migration from elasticsearch to opensearch. I am using the non-oss version of filebeat and the oss version of logstash.

I am trying to enable and install the module ibmmq with filebeat using this command.

filebeat setup -e -E output.logstash.enabled=false -E setup.kibana.ssl.enabled=false -E output.elasticsearch.ssl.verification_mode=none -E output.elasticsearch.username=admin -E output.elasticsearch.password=admin -E output.elasticsearch.hosts=['https://os02:9200']

but am getting an error:

2023-10-09T14:42:00.157Z ERROR instance/beat.go:1027 Exiting: error loading template: failed to load template: couldn’t load template: 400 Bad Request: {“error”:{“root_cause”:[{“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”}],“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”},“status”:400}. Response body: {“error”:{“root_cause”:[{“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”}],“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”},“status”:400}

Exiting: error loading template: failed to load template: couldn’t load template: 400 Bad Request: {“error”:{“root_cause”:[{“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”}],“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”},“status”:400}. Response body: {“error”:{“root_cause”:[{“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”}],“type”:“x_content_parse_exception”,“reason”:“[1:13] [index_template] template doesn’t support values of type: VALUE_STRING”},“status”:400}

Configuration:

Relevant Logs or Screenshots:

2

@wei You’ve stated that you are trying to replicate filebeat → logstash → elasticsearch with filebeat → logstash → opensearch.
But in your command, you’ve disabled the logstash plugin.
Also, to output data from Logstash to OpenSearch, you must use the OpenSearch output plugin.

Regarding the filebeat version, 7.17.13 is not supported as per OS Documentation.

3

Hi Pablo,

According to filebeat, in order to install the module to the opensearch I need to directly connect to opensearch with filebeat first. This seems to be an issue with kibana dashboards and opensearch. I was able to get this working by manually recreating the dashboards in opensearch.