Exact Accuracy Cardinality Calculations

OpenSearch
1

Versions: Using OpenSearch 2.18 (But willing to upgrade if required)

Describe the issue:
Hi I am looking for a method to perform a cardinality aggregation with exact accuracy. Even when maxing out the precision_threshold (As mentioned in the OpenSearch docs) we still encounter inaccuracy due to the algorithm used.

As it is stated “The cardinality aggregation uses the HyperLogLog++ algorithm. Cardinality counts are typically very accurate up to the precision threshold and are within 6% of the true count in most other cases“.

Here is an example query:

GET opensearch_dashboards_sample_data_ecommerce/_search
{
  "size": 0,
  "aggs": {
    "unique_products": {
      "cardinality": {
        "field": "products.product_id",
        "precision_threshold": 40000
      }
    }
  }
}

Due to our use case we need perfect accuracy. Is there any work around method to do this?

2

@tim123 As you’ve mentioned and as per OpenSearch documentation, the cardinality is using the HyperLogLog++ algorithm, which provides an approximate count of unique values.

I’m not aware of a built-in aggregation that would return exact accuracy.

Did you consider using a composite aggregation instead?