Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.13
Describe the issue:
was using neural search and querying Shoes but Jeans is getting more score than Women’s Shoes , While when I was trying the similarity score with model directly than I was getting more score for Women’s Shoes rather than Jeans, Same case is happening with other search and mostly smaller senatances are getting higher score than full santance
Configuration:
Model :- cohere.embed-multilingual-v3
Embedding field :-
"description_embedding": {
"type": "knn_vector",
"dimension": 1024,
"method": {
"engine": "lucene",
"space_type": "l2",
"name": "hnsw",
"parameters": {}
}
}
"name_embedding": {
"type": "knn_vector",
"dimension": 1024,
"method": {
"engine": "lucene",
"space_type": "l2",
"name": "hnsw",
"parameters": {}
}
Also I was doing couple of experiments and I was finding that with small data set results are good but with larger data set I’m facing this issue, So is there any way to handle that issue ?