Why my OpenSearch vector search is slow

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 1.3 on AWS

Describe the issue:
I have a OpenSearch Service instance on AWS with 2 nodes, I created an index with a knn field. My queries took 10 seconds.
The index contains more than 2 millions of data with size of 11gb.

How to improve the knn search?

Configuration:
Schema:

{
  "settings": {
    "index": {
      "number_of_shards": 2,
      "number_of_replicas": 0,
      "knn": true,
      "knn.algo_param.ef_search": 512
    }
  },
  "mappings": {
    "dynamic": "false",
    "properties": {
      "title": {
        "type": "text"
      },
      "text": {
        "type": "text"
      }
      "vector": {
        "type": "knn_vector",
        "dimension": 256,
        "method": {
          "name": "hnsw",
          "space_type": "l2",
          "engine": "nmslib",
          "parameters": {
            "ef_construction": 2000,
            "m": 16
          }
        }
      }
    }
  }
}

Query:

{
  "highlight": {
    "fields": {
      "text": {}
    },
    "fragment_size": 1000,
    "fragmenter": "simple",
    "highlight_query": {
      "match": {
        "text": "Why my OpenSearch vector search is slow?"
      }
    },
    "no_match_size": 1000,
    "number_of_fragments": 5,
    "post_tags": "</em>",
    "pre_tags": "<em>"
  },
  "_source": [
    "title",
    "text"
  ],
  "size": 100,
  "query": {
    "knn": {
      "vector": {
        "k": 100,
        "vector": [0.06414795,-0.033294678,...,-0.06677246]
      }
    }
  }
}

Relevant Logs or Screenshots:

2

I installed OpenSearch 2.9 on a local server, the vector index contains 3.5 millions of data with size of 49gb.

A simple knn search takes 8 seconds, a knn search with highlighting takes 18 seconds. We are disappointed on the search performance, compared to other vector search engines (e.g. Weaviate).
How to improve the knn search performance?