OpenSearch returns less number of search results than size

General Feedback
1

I’m using Python to index and search via OpenSearch. However, I just found out I’m getting less number of search results than the set body size. I’ve set the retrieved result size as 30 but I’m only seeing fewer than 30 search results for some queries.

Here’s my query.

phrase_query = {
    "multi_match": {
        "query": query_text, 
        "fields": ["document", "title"]
    }
}

body = {
    "size": 30,
    "query": phrase_query,
    "_source": {
        "includes": ["document", "title"]
    }
}

res = self.es.search(index=index_name, body=body, request_timeout=20)

I’d expect OpenSearch to return 30 results sorted by the score but it seems like there’s some sort of filtering going on right before returning the search results. Any help would be appreciated!

2

Hi @puff2fakie

How many hits there were in total?
You can learn this by exploring the hits.total part of the search response. Is it possible that it was less than 30?

Regards,
Lukáš

3

Hi @lukas-vlcek

Yes, the total number of hit.total was less than 30. I’m thinking other documents are calculated to be irrelevant (meaning having a BM25 score 0) and items with a score 0 is automatically filtered. However, it’s important for my system to retrieve exactly 30 results. Is there a way for me to still output 30 results in this case?

4

Hi @puff2fakie

are you able to reproduce this issue with example set of documents that you could share with us?
Do you have examples of documents that should be matching but the query does not find them?

Regards,
Lukáš

5

I am thinking you may want to use the “should” query (instead of multi_match) with a size of 30, that should bring any 30 documents that exist, sorted by their relevance score.