Describe the issue:
There seems to be no support to limit the size of docs retrieved per shard.
When is it useful?
Search index is large, assume it is an index of 500m records and the index is split into 96 shards. Each shard is ~45gb. The current implementation would fetch size(100) from each shard and so 96*100=9600 rows merge and return 100 only. We can do this better if there was shard.size param support.
The general idea is that searches for the top N most relevant documents can only do so correctly by fetching N documents from each shard. While unlikely, it’s theoretically possible that the most relevant documents all come from a single shard.
The good news is that OpenSearch doesn’t actually send 9600 documents back to the coordinator. Instead, it sends back 9600 (docId, score) tuples (or possibly (shard, docId, score)). Once it collates down to the top 100, it issues a “fetch” request to get the documents themselves from the relevant shards.