for a learning project I have set up an OpenSearch database and search it with kNN searches to find similar documents. Technically, this all works well, but it is a bit slow because I have to transport the vectors (length of 1000) twice over the network. The programme workflow looks like this:
- Select a document based on its _id and load the vectors into the client’s programme
- Do a similarity search with kNN search with the vector of the previous selected document
This step currently runs in two requests, whereby the vectors need a lot of time to be transported over the network. Therefore, I wonder if it is not possible to combine both requests with aggregation so that the vectors do not have to be sent.
- Is such an aggregation possible?
- Is something like this more efficient? - I think so, but I am not an expert in this field.
I am grateful for any help! - Have a nice day