Prometheus Catalog doesn't support aggregations without span expression

Plugins Observability
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.10 for OS, OS dashboards and all plugins, OS is Amazon linux, browser Chrome

Describe the issue:
any query I try to visualize fails with ‘Prometheus Catalog doesn’t support aggregations without span expression’

with the example here i am querying a metric provided by the postgres exporter into prometheus
whats shown from “stats” onward is generated by the UI: source=prometheus.pg_up | stats max(@value) as test by instance

as a side note, clicking table view shows lots of data for this metric

Relevant Logs or Screenshots:

image
image925×501 19.4 KB

goal is just to get some sort of visualization to work when using a prometheus metric

2

image
image2549×1228 182 KB

small log excerpt is

[2023-12-06T19:06:08,485][INFO ][o.o.s.p.PPLService       ] [opensearch-master] [3b3b44bc-4bd3-4329-b9eb-f198e330779d] Incoming request source=prometheus.pg_up | where @timestamp >= *** and @timestamp <= *** | stats max(@value) by instance | stats count(null) | fields + null
[2023-12-06T19:06:08,486][INFO ][o.o.s.p.PPLService       ] [opensearch-master] [7a3ca764-627b-4965-8a34-f185a5ceb4c3] Incoming request source=prometheus.pg_up | where @timestamp >= *** and @timestamp <= *** | stats max(@value) by instance | fields + null
[2023-12-06T19:06:08,514][INFO ][o.o.s.p.PPLService       ] [opensearch-master] [0f7207d1-e1b9-48c5-9293-b729d40490e8] Incoming request source=prometheus.pg_up | fields + null
[2023-12-06T19:06:08,539][INFO ][o.o.s.p.PPLService       ] [opensearch-master] [dd083c42-feef-44d5-b8eb-6c3dcaf79c71] Incoming request source=prometheus.pg_up | where @timestamp >= *** and @timestamp <= *** | stats max(@value) by instance | stats count(null),take(instance) by patterns_field | fields + null
[2023-12-06T19:06:08,554][INFO ][o.o.s.p.PPLService       ] [opensearch-master] [6acd7c08-b0cb-4701-9d4e-ed1dcfa455df] Incoming request source=prometheus.pg_up | where @timestamp >= *** and @timestamp <= *** | stats max(@value) by instance | stats count(null) by patterns_field | fields + null
[2023-12-06T19:06:08,566][ERROR][o.o.s.p.r.RestPPLQueryAction] [opensearch-master] Error happened during query handling
java.lang.RuntimeException: Prometheus Catalog doesn't support aggregations without span expression
3

I can get things to work if I stay away from the UI buttons and manually build the query. using query workbench worked best here. I then go back and paste it in. For example:
source = prometheus.pg_up | stats avg(@value) by span(@timestamp,1h)
works just fine.

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.