Hi,
We are using rollup to aggregate data in a rollup index. Unfortunately, this index are not user friendly. For instance we are not able to perform some aggregation in visualization such as “Missing”, “Other”.
We are not able to perform rollover on those index as well.
I guess this is because rollup index is not an index as others, and we are not able to query it like any other index. Why ? Is it not possible to create a normal index for rollup ?
Thanks,
Julien
Hey @Julien,
I believe the “missing” is something we didn’t get to yet for the terms aggregation. We do plan to support it. What is the “Other” you’re referring to?
What is the error you’re seeing for rollover?
We do treat it as a somewhat special index, but it is just an index. The only thing we do extra is add a setting that specifies it contains rolled up data so we can intercept search requests to the index and transform them. The reason we do this is because the data in your original source index and the data in the rolled up index are stored in different formats. i.e. you might have "some_number": <number> in a bunch of your original documents, but when you do max, min, sum, avg, etc. on that field we need to store a bunch of different values.
We wanted to allow users to query the original and rolled up index with the same queries though, so what we do is intercept search requests and transform them to match the new format.
Hi @dbbaughe ,
I mean “Show missing values” and “Group other values in separate bucket” in visualization for a split. Count is not available either, we have to do a Sum aggregation on an event.count field that we created.
It is also impossible to search this index in Discover. This limit is coming from search query, we have to specify size = 0 in the search query to get result, but then we do not have hits. And this is why, unfortunately, number of features are not available on rollup index.
Rollover does not work because this index is not based on an alias. Am I wrong ? delete_by_query does not work neither.
Thanks