Issue:
I am using “PutElasticSearchHttp” processor in Nifi which is not sending data in opensearch. It is throwing bad Request error.
I have checked my opensearch IP and ports are correct , cluster is also healthy but data is not being ingested.
Relevant Logs:
Following error is coming in logs of Nifi
WARN [Timer-Driven Process Thread-5] o.a.n.p.e.PutElasticsearchHttp PutElasticsearchHttp[id=f839f409-8844-3c80-91e9-166029e9c813] Elasticsearch returned code 400 with message Bad Request, transferring flow file to failure
@Iqra_shafiq I’m not familiar with Apache Nifi, however, did you check the OpenSearch’s logs against any connectivity errors?
What user did you use in Apache Nifi to connect with the OpenSearch cluster?
@Iqra_shafiq Disabling SSL doesn’t disable authentication/authorization but only secured communication on port 9200. You’re still required to authenticate with OpenSearch.
Only disabling the security plugin will disable authentication but I don’t recommend this approach.
@Iqra_shafiq Can you trace Apache Nifi’s APIs sent to the OpenSearch cluster?
If yes, have you tried to send the PUT API constructed by Apache Nifi manually?