Neural Sparse Search and bucket aggregation filtering

OpenSearch
1

Hi,

We are using neural sparse search and we want to keep the best x% matches. To do this, we first run the query, get the max score and then run the query again, adding min_score = x% * max_score. All the filtering clauses (ranges, terms, etc) are in a filter context and just the neural_sparse clause is in a query context (must) so the score is affect just by the sparse search.

However, the problem is that we want to aggregate the data based on a term and we want to keep the top x% results per bucket, not top x% of all the results returned by the initial filtering.

What we have now: filter → keep top x% based on _score → split into buckets
What we would want: filter → split into buckets → keep top x% per bucket based on the _score inside the bucket

How can this be achieved?

Thanks!

Note: Using AOS 2.11

2

Hi @grunt-solaces.0h , is the min_score used as filter context in the second query? Or you want to calculate the min_score for each bucket?

3

Have you taken a look at the collapse feature? Where we can collapse the search result on a field to get buckets and hits on each buckets?