After Upgrade Opensearch 2.6.0 fluentbit gives error: Action/metadata line [1] contains an unknown parameter [_type]"},"status":400

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
We are using Fluentbit as a ingestion tool for Opensearch .

Previously: Everything working fine
Opensearch 1.3.7
Fluentbit: 1.6 (using elasticserach plugin)

After upgrade Opensearch from 1.3.7 to 2.6.0
Opensearch: 2.6.0
Fluentbit: 1.6

fluentbit start giving error :
[output:es:es.0] HTTP status=400 URI=/_bulk/?pipeline=my_pipeline, response:
{“error”:{“root_cause”:[{“type”:“illegal_argument_exception”,“reason”:“Action/metadata line [1] contains an unknown parameter [_type]”}],“type”:“illegal_argument_exception”,“reason”:“Action/metadata line [1] contains an unknown parameter [_type]”},“status”:400}

Describe the issue:
fluentbit start giving error :
[output:es:es.0] HTTP status=400 URI=/_bulk/?pipeline=my_pipeline, response:
{“error”:{“root_cause”:[{“type”:“illegal_argument_exception”,“reason”:“Action/metadata line [1] contains an unknown parameter [_type]”}],“type”:“illegal_argument_exception”,“reason”:“Action/metadata line [1] contains an unknown parameter [_type]”},“status”:400}

We don’t want to change anything on flunetbit side, is there any way we can handle this in Opensearch only.

Already tried :
“compatibility.override_main_response_version”: true
but no success.

Configuration:

Relevant Logs or Screenshots:

{“error”:{“root_cause”:[{“type”:“illegal_argument_exception”,“reason”:“Action/metadata line [1] contains an unknown parameter [_type]”}],“type”:“illegal_argument_exception”,“reason”:"Action/metadata line [1] contains an unknown parameter [_type]

2

I wouldn’t expect this to work without changing the FluentBit side. We follow semantic versioning which in this case means we make breaking changes to the API’s between OpenSearch 1.x line (which was fully compatible with Elasticsearch 7.10.x) and the OpenSearch 2.x line.

In this case the _type field was removed which is causing the break you are experiencing.

3

Yeah, you’ll probably need to set Suppress_Type_Name to On in the output. But you’ll need FluentBit 1.7 for it: Elasticsearch - Fluent Bit: Official Manual