Can I use _reindex api to copy more than 10000 documents

OpenSearch
1

OpenSearch 2.11

I plan to use _reindex API to copy subset of one index to another. I am not interested in specific documents, can by random set.

By making the _reindex request with max_docs and size set over 10000, I get following error.

"error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Batch size is too large, size must be less than or equal to: [10000] but was [20000]. Scroll batch sizes cost as much memory as result windows so they are controlled by the [index.max_result_window] index level setting."
      }
    ],

Is there a way to copy more, but not all?
The index is very simple, and has three fields, I use query to narrow what documents to use:

"query": {
      "term": {
        "customer_id": 12345
      }
    }

I understand the concern and error. But is it possible to make OpenSearch scroll thru query results automagically?

2

Oh, never mind - probably :slight_smile:
The problem was of size field in my request:

{
  "max_docs": 10000,
  "conflicts":"proceed",
  "source": {
    "size": 10000,

Removing the size, but keeping max_docs seem to help.