Knn search is too slow

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

Opensearch version - 2.5

Describe the issue:

Tried all three query examples showed here - k-NN - OpenSearch documentation

All of them could not return results in 30s.

My index has 33M documents, with some of the documents containing knn_vector fields. Vectors is of 384 dimensions. Below is index settings which I used to create it.

{
    "index_name": {
    "aliases": {},
    "mappings": {
      "properties": {
        "embedding_vector": {
          "type": "knn_vector",
          "dimension": 384,
          "method": {
            "engine": "nmslib",
            "space_type": "l2",
            "name": "hnsw",
            "parameters": {
              "ef_construction": 128,
              "m": 24
            }
          }
        }
    },
    "settings": {
      "index": {
        "refresh_interval": "300s",
        "number_of_shards": "10",
        "provided_name": "index_name",
        "knn": "true",
        "creation_date": "",
        "number_of_replicas": "0",
        "uuid": "",
        "version": {
          "created": ""
        }
      }
    }
  }
}

One of the example queries I am trying:

{
  "size": 20,
  "query": {
    "script_score": {
      "query": {
        "match_all": {}
      },
      "script": {
        "source": "knn_score",
        "lang": "knn",
        "params": {
          "field": "company_description_embedding",
          "query_value":[...]
          "space_type": "l2",
          "k":20
        }
      }
    }
  }
}

Running cluster on aws, c6g.4xlarge.search instance, 1 node, 500G volume.
Ran out of ideas what to try out. It seems, that those vectors were somehow not indexed or something like that. Do you have any ideas/suggestions what can I check/try?

2

I’m late to the party, but maybe some of the following would help: