Terms search gives error "failed to create query: maxClauseCount is set to 1024"

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.x

Describe the issue: I am using terms search with index and document id. My query looks something like this:

{
  "query": {
    "terms": {
      "ip": {
        "index": "address_index",
        "id": "b2e29f44-ef98-4426-9bfb-34881a839cdc",
        "path": "ip_values"
      }
    }
  }
}

I have more than 10K values for this document in my address_index.

Note: Query works If I use another document ID where I have less than 1024 values.

Configuration:

Relevant Logs or Screenshots:

    "reason": {
      "type": "query_shard_exception",
      "reason": "failed to create query: maxClauseCount is set to 1024",
      "index": "address_index",
      "index_uuid": "4oi75d-kk45pgpEPvA",
      "caused_by": {
        "type": "too_many_clauses",
        "reason": "maxClauseCount is set to 1024"
      }
    }

Can you run _analyze API if it’s possible to modify indices.query.bool.max_clause_count from the default(1024) to 2048 or above?

You must be careful that this setting will have an impact on memory usage in clusters so it’s just for checking how many clauses are created by the query you ran. After you test, now’s the time to determine the adequate number for max_clause_count.

As you know, OpenSearch’s _search queries are always rewritten to Lucene’s BooleanQuery internally.

Thanks @yeonghyeonKo

I am migrating from elasticSearch 7.10 and this same query works there without changing default value of indices.query.bool.max_clause_count. I am really not sure to modify max_clause_count because my document could have any number of values. I am trying to understand why am I getting this error with openSearch?

stacktrace would be very helpful to confirm the reason. For awhile, I remember that some of the “multiterm queries” support rewrite property. Could you try to pass "rewrite":"constant_score" ?
UPD pardon, there are no rewrite field in termsquery OpenSearch/server/src/main/java/org/opensearch/index/query/TermsQueryBuilder.java at main · opensearch-project/OpenSearch · GitHub

Thanks @mkhl

The only stacktrace I am getting from opensearch

  "took": 50,
  "timed_out": false,
  "_shards": {
    "total": 63,
    "successful": 56,
    "skipped": 0,
    "failed": 7,
    "failures": [
      {
        "shard": 0,
        "index": "address_index",
        "node": "EhLUz-OFQEDSEQK1-kjnfg",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to create query: maxClauseCount is set to 1024",
          "index": "address_index",
          "index_uuid": "4oi73eMuSX-qw23pgpEPvA",
          "caused_by": {
            "type": "too_many_clauses",
            "reason": "too_many_clauses: maxClauseCount is set to 1024"
          }
        }
      }```

Common REST Parameters - OpenSearch Documentation here’s how you can get a stacktrace. It looks like a feature request I want to tackle.

Thanks @mkhl

I got the stack trace now.

        "type": "too_many_clauses",
        "reason": "too_many_clauses: maxClauseCount is set to 1024",
        "stack_trace": "NotSerializableExceptionWrapper[too_many_clauses: maxClauseCount is set to 1024]\n\tat org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:115)\n\tat org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:129)\n\tat org.opensearch.index.mapper.MappedFieldType.termsQuery(MappedFieldType.java:237)\n\tat org.opensearch.index.mapper.IpFieldMapper$IpFieldType.termsQuery(IpFieldMapper.java:257)\n\tat org.opensearch.index.query.TermsQueryBuilder.doToQuery(TermsQueryBuilder.java:476)\n\tat org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)\n\tat org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)\n\tat org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:328)\n\tat org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)\n\tat org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)\n\tat org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:326)\n\tat org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)\n\tat org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)\n\tat org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:329)\n\tat org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)\n\tat org.opensearch.index.query.QueryShardContext.lambda$toQuery$3(QueryShardContext.java:467)\n\tat org.opensearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:479)\n\tat org.opensearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:466)\n\tat org.opensearch.search.SearchService.parseSource(SearchService.java:1238)\n\tat org.opensearch.search.SearchService.createContext(SearchService.java:985)\n\tat org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:593)\n\tat org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:566)\n\tat org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74)\n\tat org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89)\n\tat org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)\n\tat org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78)\n\tat org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)\n\tat org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59)\n\tat org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908)\n\tat org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.lang.Thread.run(Thread.java:833)\n"
      }```

Posting it in more readable view. Wondering why TermsQuery wasn’t rewritten to bitset. It’s worth an issue for sure.

"type": "too_many_clauses",
        "reason": "too_many_clauses: maxClauseCount is set to 1024",
        "stack_trace": "NotSerializableExceptionWrapper[too_many_clauses: maxClauseCount is set to 1024]
	at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:115)
	at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:129)
	at org.opensearch.index.mapper.MappedFieldType.termsQuery(MappedFieldType.java:237)
	at org.opensearch.index.mapper.IpFieldMapper$IpFieldType.termsQuery(IpFieldMapper.java:257)
	at org.opensearch.index.query.TermsQueryBuilder.doToQuery(TermsQueryBuilder.java:476)
	at org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)
	at org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)
	at org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:328)
	at org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)
	at org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)
	at org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:326)
	at org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)
	at org.opensearch.index.query.BoolQueryBuilder.addBooleanClauses(BoolQueryBuilder.java:346)
	at org.opensearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:329)
	at org.opensearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:117)
	at org.opensearch.index.query.QueryShardContext.lambda$toQuery$3(QueryShardContext.java:467)
	at org.opensearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:479)
	at org.opensearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:466)
	at org.opensearch.search.SearchService.parseSource(SearchService.java:1238)
	at org.opensearch.search.SearchService.createContext(SearchService.java:985)
	at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:593)
	at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:566)
	at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74)
	at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59)
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(Thread.java:833)
"
      }```

here IpFieldMapper it falls back to MappedFieldType which is prone to maxClauses error. Presumably if you avoid slashes in those IPs it should work.

UPD Do you have an idea how many concrete IPs and mask/s you have in those lists?
What can work is if you have a separate fields for concrete IPs and mask/s - in this case concrete IPs isn’t limited by max_caluses, but mask/s still encounter this limitation. Then you can bool/should two term lookups. It’s not ideal workaround but this may work.

raised [Feature Request] relax max Clauses Count limitation of termS query over IP field · Issue #16200 · opensearch-project/OpenSearch · GitHub