Create visualisation of data in rollup index

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch and OpenSearch Dashboard version 2.8.0 (hosted on Aiven, doubt that’s relevant)

Describe the issue:

I’ve got some indexes containing time-series data. I’ve got an ISM policy that, after three days, performs a rollup on the data to a single destination index.

For the original source indeces I’m able to create an index pattern and create visualisations and assemble them into a dashboard for the live data.

I want to create an equivalent dashboard for the rollup data but this is where I’m having problems. In order to create a visualisation I need an index pattern to use as the source, but I’m unable to create an index pattern for the rollup index. The backend is sending an empty reply to one of the calls in the background. Copying the failing request as curl, gives me something like the following (trimmed for readability)

➜  ~ curl 'http://localhost:7000/api/index_patterns/_fields_for_wildcard?pattern=my-rollup-index-2023&meta_fields=_source&meta_fields=_id&meta_fields=_type&meta_fields=_index&meta_fields=_score' \
       -H 'Accept: */*' \
       -H 'Accept-Language: en-US,en;q=0.9,no;q=0.8' \
       -H 'Authorization: Basic <SECRET>' \
       -H 'Content-Type: application/json' \
       --compressed
curl: (52) Empty reply from server

Is this supposed to work for rollup indexes? If not how do I create the visualisation over my rollup data? I feel like I’m missing something obvious here.

I see that Elastic/Kibana has a dedicated index pattern type for rollup indexes but I find no such similar functionality in OpenSearch Dashboards.

1 Like
2

Just FYI inspecting the logs doesn’t give me much info

The opensearch process just logs

Has rollup indices will rewrite field caps response

and the OpenSearch Dashboards logs that it’s returning a 404 response after 30 seconds

 404 30003ms - 9.0B
3

Right, after a little more investigation it seems that the Empty reply from server is caused by some proxy server with a timeout. I’ve tried increasing all the timeouts I have access to but there’s one proxy I can’t affect that has the timeout set to 60seconds.

That said, my rollup index in this instance has a total size of 120kb … I struggle to understand what could be taking longer than 60 seconds here…