# Timeouts during the search load test

**URL:** <https://forum.opensearch.org/t/timeouts-during-the-search-load-test/11911>\
**Category:** k-NN\
**Created:** [December 22, 2022, 5:37pm UTC](https://forum.opensearch.org/t/timeouts-during-the-search-load-test/11911 "2022-12-22T17:37:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![PaulNarbe](https://avatars.discourse-cdn.com/v4/letter/p/f07891/32.png) [@PaulNarbe](https://forum.opensearch.org/u/PaulNarbe)\
**Post date:** [December 22, 2022, 5:37pm UTC](https://forum.opensearch.org/t/timeouts-during-the-search-load-test/11911/1 "2022-12-22T17:37:25Z")

</div>

**Versions**  
Opensearch 2.3

**Describe the issue**  
We implemented a search queries load test for the OpenSearch cluster with the knn plugin installed.  
During the load test we get a lot of timeouts shortly after the start.  
Index with ~22M items is pre-uploaded (144Gb storage, 32 shards, 890 segments). The vectors inside are 512-d, lucene hnsw is used.  
ef\_construction: 32  
M: 64  
load test concurrency: 25  
**What is the expected behavior?**  
Either timings are low or any watched metrics shows clearly the reason of the problem (something to be scaled or reconfigured).  
**What is your host/environment?**  
The cluster is run on the 16 m6g.xlarge.search data nodes and 3 r6g.large.search master nodes.  
**Do you have any additional context?**  
We’re trying to monitor the source of the problem using CPUUtilization, JVMMemoryPressure, Free Storage.  
None of those gets close to the limit during the test.  
KNNGraphMemoryUsage is always 0, which is different from faiss and nmslib hnsw tests.  
Can you please give me any guidance on what metrics or potential problems should I look for?

---

<div class="post-metadata">

**Author:** ![martin.g](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/martin.g/32/5294_2.png) [@martin.g](https://forum.opensearch.org/u/martin.g)\
**Post date:** [January 4, 2023, 9:59pm UTC](https://forum.opensearch.org/t/timeouts-during-the-search-load-test/11911/2 "2023-01-04T21:59:57Z")

</div>

Hello,

What is your timeout setting (I assume you got timeouts for query load type), value of k, amount of RAM allocated for JVM on data nodes, num of replicas?

I can give few general recommendations after looking on provided description:  
try values of m and ef\_construction that are closer to Lucene defaults: m = 16, ef\_construction = 100.  
I’m not sure if you merge segments or not, but with your settings 890/32shards ~28 segments per shard can be a lot. try lower number of segments to say 10 per shard, you can call force\_merge from Index API with max\_segments = 10 (number of segments is per shard). More segments give you better recall but tradeoff is higher latency as search times per segment are combined and sum is greater than search time for a single big segment.