Aggregations.composite_buckets.composite.size:1000 seems hardcoded for GROUP BY

Plugins SQL
1

Hi!
When I run query with aggregation using GROUP BY I only get 1000 results because of aggregations.composite_buckets.composite.size:1000 inserted by parser:

POST /_plugins/_sql/_explain
{
  "query": """
  SELECT Account, SUM(Credits) FROM someevents-* GROUP BY Account
  """
}

{
  "root": {
    "name": "ProjectOperator",
    "description": {
      "fields": "[Account, SUM(Credits)]"
    },
    "children": [
      {
        "name": "OpenSearchIndexScan",
        "description": {
          "request": """OpenSearchQueryRequest(indexName=someevents-*, sourceBuilder={"from":0,"size":0,"timeout":"1m","aggregations":{"composite_buckets":{"composite":{"size":1000,"sources":[{"Account":{"terms":{"field":"Account","missing_bucket":true,"missing_order":"first","order":"asc"}}}]},"aggregations":{"SUM(Credits)":{"sum":{"field":"Credits"}}}}}}, searchDone=false)"""
        },
        "children": []
      }
    ]
  }
}

I want to increase amount of results returned. Is it even possible? Per request to SQL API or by cluster setting.
I looked at _cluster/settings but couldn’t find anything related. LIMIT 2000 not helping too.

Thanks

2

Yes, it’s a limitation of the current plugin. Bucket size is hard-coded here.

I believe this is the issue to address it.

1 Like