# How to separate ml ingestion workload from search workload

**URL:** <https://forum.opensearch.org/t/how-to-separate-ml-ingestion-workload-from-search-workload/16532>\
**Category:** Machine Learning\
**Created:** [November 2, 2023, 1:43pm UTC](https://forum.opensearch.org/t/how-to-separate-ml-ingestion-workload-from-search-workload/16532 "2023-11-02T13:43:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![asfoorial](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@asfoorial](https://forum.opensearch.org/u/asfoorial)\
**Post date:** [November 2, 2023, 1:43pm UTC](https://forum.opensearch.org/t/how-to-separate-ml-ingestion-workload-from-search-workload/16532/1 "2023-11-02T13:43:17Z")

</div>

OS 2.11.0

How can I seperate ML ingestion workload and search workloads on the same index. I have 2 master nodes, 2 data nodes and 2 (ingest+ml) nodes.

I tried to create both sparse and dense indices and I noticed very high CPU utilization during ingestion. This has tremendously lowered search speed over the index.

Any recommendations?

Also, is there a way to force ingestion pipelines to use specific ML nodes so that I always have free ML nodes for search workloads?

Thanks

---

<div class="post-metadata">

**Author:** ![ylwu](https://avatars.discourse-cdn.com/v4/letter/y/a9a28c/32.png) [@ylwu](https://forum.opensearch.org/u/ylwu)\
**Post date:** [November 18, 2023, 1:07am UTC](https://forum.opensearch.org/t/how-to-separate-ml-ingestion-workload-from-search-workload/16532/2 "2023-11-18T01:07:34Z")

</div>

hi, @asfoorial , you can set this setting as true `plugins.ml_commons.allow_custom_deployment_plan`, then you can deploy model to a specific node. That way can help shift some ML model workload. But the ingestion and searching workload seems hard to split as they depend on the same index.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [January 17, 2024, 1:08am UTC](https://forum.opensearch.org/t/how-to-separate-ml-ingestion-workload-from-search-workload/16532/3 "2024-01-17T01:08:19Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.