_update_by_query returns version_conflict_engine_exception with single request

OpenSearch
1

Versions: 3.7.0

Describe the issue:

Hello folks!

  • Run local cluster using the sample docker-compose from docs Docker | OpenSearch Documentation
  • Load data that exceeds the configured memory limit, for example, а 1Gb index
  • Start \_update_by_query operation from Dev Tools

The operation returns plenty of version_conflict_engine_exception errors. If the index size is smaller, the query runs just fine.

Questions:

  • Why does this happen?
  • What process modifies the index during update? I’m running only a single request on a local cluster and doing nothing else.
  • How can I debug this?
2

Hi @Smoking_Machine, welcome to the forum!

Could you share your full _update_by_query query?

3

Hello!

My query looks like

POST test-idx-1/_update_by_query
{
  "query": {
    "term": {
        "groups.billing_group.keyword": "forwarding"
    }
  },
  "script": {
    "source": "ctx._source.groups.billing_group = params.grp",
    "lang": "painless",
    "params": {
      "grp": "forwarding2" 
    }
  }
}
4

@Smoking_Machine Did you consider using sroll_size and slices?
I’ve tested with a 3 GB index and 46K documents and I’ve got all documents updated without version conflicts.

i.e.

POST /test-logs/_update_by_query?slices=10&scroll_size=100&conflicts=proceed&refresh=true
{
  "query": {
    "match_all": {}
  },
  "script": {
    "source": "ctx._source.test_update_counter = 5"
  }
}