Approx neighbor query is very slow

Plugins k-NN
1

I have about 3.5M documents (approx index size 90G and it can grow upto 10x this size). I have a knn_vector field A which is of 5000 dimensions and one other field B with 80 dimensions. I use the approx neighbors query as part of my filter query and that part for A takes about 60 to 90 seconds. when I replace the same query with the exact search of A, I get sub 200ms response time.
From what I observe, the exact search via script for A is much faster (<200ms) when compared to the approx search which is the opposite of what is expected.

When the same operations are performed for B, the results are as expected and latency is just about 30ms.
I run this on a i3.2xlarge machine and have 10 shards spread across 3 nodes (Initially had 3 shards and increased up to 10 to see if there are improvements).

Sample query:

"query": {
    "function_score": {
        "query": {
          "bool": {
            "filter": [
              {
                "term": {
                  "filter1": "value1"
                }
              },
               {
                "knn":{
                  "A": {
                      "vector": [1,2,3...5k],
                       "k": 100
                       }
                }],
               functions: [
                  {
                     "script_score": {
                            "script": {
                               "lang": "knn",
                               "source": "knn_score",
                               "params": {
                                    "field": "A",
                                    "vector": [1,2,3,...5k],
                                    "space_type": "l2"
                                }
                           }
                      }
                  }
           ]
         }
       }
     }
}

The above example is a simplified version of what I’m trying to do. The exact search (script_score) will also have other features and we want to control the score by adjusting weights for each function. The plan was to do approximate search on the 5k dim vector and use the other features to boost but we didn’t expect this to happen.Can someone throw some light on what I need to do next and where to look?

2

@arvindmsd Sorry for responding late. i am looking at your use case and will get back to you soon.

3

@arvindmsd In general you can try these methods (Performance Tuning - Open Distro Documentation) to tune your performance . Moreover, it is not recommended to have filter query and a-knn query together unless you have a high k value. This is because, a-knn will be performed on complete dataset and filter will be applied on top of it. If you use exact search, this will be applied after your filter is succeeded. Hence, the number of dataset to be searched depends on your filter query. I would guess that your term filter might reduce significant dataset from consideration in total. You can verify this by performing the search with term filter alone and see number of datasets on which exact knn search is performed. Please let us know if you still face this performance issue after tuning as recommend in our doc page.

4

Hi @Vijay
The reason for having an ANN query in filter context is that we would like to reduce the search space for exact search. My assumption was that the filters get executed in the same order and hence approximation happens after the first term filter. Is that not the case?
Also we need the exact search in order to get the scores and boost by that feature. The issue I was mentioning seems to be a provisioning issue and has disappeared when we opted for a bigger instance.

Edit: I just went back to check how it works. Looks like the ANN is taking precedence as compared to other filters. Is there any way to solve this?

This is because, a-knn will be performed on complete dataset and filter will be applied on top of it.

Is there a way for us to determine which subsets can be neighbors during index time? Shard routing maybe?

5

@arvindmsd The limitation with building ANN graph at index time is that it can’t be applied on top of filtered data, since search happens independent of other filter by the library, purely based on graph. Hence, we have custom filtering (brute force search) which is not ANN for this scenario. I don’t think routing will help, i can check and get back to you.

Also, it is not ANN is taking precedence, it is the implementation that ANN is only possible on graph that was built during index time.

6

Understood. It seems to do an intersection with the filters.

7

@arvindmsd did you solve this ? i’m facing the same issue.