Should The Same FAISS k-NN Query Yield The Same Results Each Time?

Plugins k-NN
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.9

Describe the issue:
I am using OpenSearch k-NN with FAISS (HNSW method). I am finding that without changing my underlying index, the same k-NN query yields different results (i.e. it is non-deterministic).

I would have thought that any non-determinism would be in the indexing path and not in the query path. Is this non-determinism expected? Note that I am using efficient pre-filtering, which could potentially also contribute to non-determinism.

Another interesting fact: when I change k to be much greater than size, I find that the non-determinism goes away.

Here’s an example query:

GET my_index/_search
{
 "size": 50,
 "fields": ["_id"],
 "_source": false,
 "query": {
   "knn": {
     "my_knn_field": {
       "k": 1000,
       "filter": {
         "bool": {
           "must": [{
             "terms": {
               "some_field": [123, 456, 789]
             }
           }]
         }
       },
       "vector": [1, 2, 3, 4, 5]
      }
    }
  }
}
1 Like