_plugins/_asynchronous_search API does not return partial results for non aggregation query

OpenSearch
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.15.0

Describe the issue:
I am expecting to see partial results returned when using _plugins/_asynchronous_search endpoint for non aggregation query

POST _plugins/_asynchronous_search/?pretty&wait_for_completion_timeout=1ms&keep_on_completion=true&request_cache=false&index=test-many-shards&track_total_hits=true
{
  "size": 50,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "ingestDate": {
              "gte": "now-168h",
              "lte": "now"
            }
          }
        }
      ]
    }
  }
}

But looks like it only return the last final result without the intermediate results. Is this the expected behavior in OS? (I have no issue using _plugins/_asynchronous_search/ API to run aggregation query (e.g. with “aggs” field set) and see the partial results get returned at shard level.)

Configuration:
GET _cat/indices/test-many-shards?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open test-many-shards zy2KfC2qRqOSpR-ZMCkZMw 32 1 17457803 3478897 63.5gb 31.8gb

Relevant Logs or Screenshots:
Some in progress response from _plugins/_asynchronous_search for id FldnODFXRzBhUzRhYjdfbXZFR0tkcVEFNzk0NjAUZkFVazNKQUJOMXVod0dicWJ0Q0QCMjE=

{
  "id": "FldnODFXRzBhUzRhYjdfbXZFR0tkcVEFNzk0NjAUZkFVazNKQUJOMXVod0dicWJ0Q0QCMjE=",
  "state": "RUNNING",
  "start_time_in_millis": 1721680293507,
  "expiration_time_in_millis": 1721766693507,
  "response": {
    "took": 20483,
    "timed_out": false,
    "num_reduce_phases": 3,
    "_shards": {
      "total": 32,
      "successful": 15,
      "skipped": 0,
      "failed": 0
    },
    "hits": {
      "total": {
        "value": 5049140,
        "relation": "eq"
      },
      "max_score": null,
      "hits": []
    }
  }
}

{
  "id": "FldnODFXRzBhUzRhYjdfbXZFR0tkcVEFNzk0NjAUZkFVazNKQUJOMXVod0dicWJ0Q0QCMjE=",
  "state": "RUNNING",
  "start_time_in_millis": 1721680293507,
  "expiration_time_in_millis": 1721766693507,
  "response": {
    "took": 40764,
    "timed_out": false,
    "num_reduce_phases": 7,
    "_shards": {
      "total": 32,
      "successful": 30,
      "skipped": 0,
      "failed": 0
    },
    "hits": {
      "total": {
        "value": 11419958,
        "relation": "eq"
      },
      "max_score": null,
      "hits": []
    }
  }
}

As you can see the “hits” field return empty list as partial results even there is some num_reduce_phases completed

The final response from _plugins/_asynchronous_search for id FldnODFXRzBhUzRhYjdfbXZFR0tkcVEFNzk0NjAUZkFVazNKQUJOMXVod0dicWJ0Q0QCMjE=

{
  "id": "FldnODFXRzBhUzRhYjdfbXZFR0tkcVEFNzk0NjAUZkFVazNKQUJOMXVod0dicWJ0Q0QCMjE=",
  "state": "STORE_RESIDENT",
  "start_time_in_millis": 1721680293507,
  "expiration_time_in_millis": 1721766693507,
  "response": {
    "took": 49259,
    "timed_out": false,
    "num_reduce_phases": 8,
    "_shards": {
      "total": 32,
      "successful": 32,
      "skipped": 0,
      "failed": 0
    },
    "hits": {
      "total": {
        "value": 12741817,
        "relation": "eq"
      },
      "max_score": 0,
      "hits": [
        {
          "_index": "test-many-shards",
          "_id": "6278716",
          "_score": 0,
          "_source": {
            "ingestDate": "2024-07-18T06:40:37.765+0000",
            "logDate": "2024-07-18T06:40:35.371+0000",
            "logTimestamp": 1721284835371,
           ...

As you can see, only the final response contains the results (not paste all records for security purpose) in “hits” field.

2

Hi OpenSearch team,

It will be great if someone can answer my simple question. I am basically expecting a Yes or No answer here. Really appreciated!

3

An engineer with years of experience in full-text search engines here; not an expert in ElasticSearch/OpenSearch, so I can’t provide a definitive “yes” or “no” answer. Instead I’ll try to explain why the behavior you’re encountering may be expected.

The decision not to provide partial document hits in the results of an asynchronous search in OpenSearch (or a similar system) can stem from design choices related to consistency and user expectations.

In aggregation queries, intermediate results from each shard can be merged to form a meaningful partial result; each shard contributes to a cumulative result even before all data is fully processed.

In contrast, for document fetches, merging partial hits from different shards isn’t straightforward and can complicate the consistency of returned results. For instance, if some shards that contain highly relevant documents have not yet completed their processing, the early results might be misleading or of lower relevance.

Users typically expect that a query will return the same results when run under the same conditions. Meanwhile, partial, non-final results from document searches could vary significantly between executions, depending on the shard response times, which might not be desirable or useful.

1 Like