Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch version: 2.11
Describe the issue:
I’m trying to combine hybrid search (semantic + keyword) with collapse feature to deduplicate products from same visual.
I have tried collapsed search on a basic search, which works great.
With hybrid search, the behaviour is a bit different. It places products from the same visual in the inner_hits field but did not collapse them (they are still present in the root level of the search results) which is not the expected behaviour.
Anyone’s aware of a problem of compatibility between hybrid and collapse ?
You can find both of the mentioned queries below
Relevant Logs or Screenshots:
Basic search with collapse (working as expected) :
GET /product_1/_search
{
“_source”: {
“includes”: [“_id”, “name”, “category_name”, “visual.id_visual”]
},
“query”: {
“match”: {
“name”: {
“query”: “Ski”
}
}
},
“collapse”: {
“field”: “visual.id_visual”,
“inner_hits”: {
“size”: 1,
“name”: “from_same_visual”,
“sort”: [
{
“_score”: “desc”
}
]
}
}
}
Hybrid search with collapse (not working) :
GET /product_1/_search?search_pipeline=nlp-search-pipeline_minmax_arithmetic_mean_55_45
{
“_source”: {
“includes”: [“_id”, “name”, “category_name”, “visual.id_visual”]
},
“query”: {
“hybrid”: {
“queries”: [
{
“neural”: {
“fullname_v”: {
“query_text”: “Ski”,
“model_id”: “QMrsMo4BK9LQeuL9rxK1”,
“k”: 200
}
}
},
{
“multi_match”: {
“query”: “Ski”,
“type”: “most_fields”,
“fields”: [“category.name^2”, “name^4”, “tags.name^3”],
“fuzziness”: “AUTO”,
“prefix_length”: 0,
“max_expansions”: 10
}
}
]
}
},
“collapse”: {
“field”: “visual.id_visual”,
“inner_hits”: {
“size”: 1,
“name”: “from_same_visual”,
“sort”: [
{
“_score”: “desc”
}
]
}
}
}