Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Describe the issue:
At opensearch-project/k-NN/pull/490, the limits for faiss and nmslib were increased.
lucene plugin limits remained aligned with
However Lucene devs, AFAICT, do not consider increasing this limit at the moment.See Increase KnnByteVectorField limit on dimensions to 2048 by benwtrent · Pull Request #12191 · apache/lucene · GitHub.
Would you folks consider increasing the
lucene plugin limits like ElasticSearch did at PR-95257?
I haven’t wrapped my head around how exactly you included the Lucene k-NN, but I am very curious if anyone could give me an indication of how I could go around patching OpenSearch to increase these limits.