Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.10
Describe the issue:
I am running the PoC cluster (eventually, we will use managed service), and during testing with moderate (IMO) load, I found some of the docs not indexed. On the client side is the Ruby HTTP client (Typhoeus), and for failed cases, it returns a 0 status code with nobody - suggesting an inability to connect.
I have background workers who execute calls to OS, and at the moment, I cannot tell the exact figure, but requests may come up to 30 at once.
I encountered the problem when attempting to index ~700 documents. ~60 of them failed due to a dropped/refused connection. The index has three fields; one is a k-nn vector (IVF index).
I did a synthetic test of sequential indexing of 5000 documents (using bulk and regular APIs), and they went flawlessly.
Can you give some pointers on how to diagnose the problem? Is there any built-in or configurable rate limit which could be the cause?
Configuration:
3 nodes x 15GB, deployed in EKS
Relevant Logs or Screenshots: