Hi, I am using OpenSearch ingest pipeline to process and parse the data before indexing. So I would like to know is there any issue with using this method to process and parse the data like performance impact or anything else?
Hi @mann , a bit of context will be helpful. There is certainly some overhead implied when using ingest pipeline, but it is highly dependent on what pipeline is doing, ingestion rate, etc. In general, for heavy ingest loads the recommendation is to have dedicated nodes with ingest role so to minimise the impact on regular cluster operations. For the lightweight workloads, it may not be necessary.
1 Like