Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 2.11
Describe the issue:
I have a live index with time series data ( alias - log ). Now I have a rollover policy which rolls up the old index into a new rollup index, and a new index is created as the current live index ( alias - log ).
Reference: Index rollups - OpenSearch documentation basically following this example.
Now I want to run a single aggregated query which aggregates on both the live index and the rolled up index. Elasticsearch supports this using the “/_rollup_search” endpoint. However, if I try to do the same on OpenSearch, I get the following error: “Not all indices have rollup job”. Note: It also returns the correct aggregated response. I wanted to know if there’s a correct way of doing this.
Configuration:
OpenSearch 2.11 ( AWS managed )
Relevant Logs or Screenshots:
search query:
curl -XPOST -H "Content-Type:application/json" "<host>/rollup_ndx-log-000001,log/_search?pretty" -d '{
"size": 0,
"query": {
"match_all": {}
},
"aggregations": {
"message_numbers": {
"terms": {
"field": "message.keyword"
},
"aggs": {
"per_message": {
"terms": {
"field": "message.keyword"
},
"aggs": {
"sum_message": {
"sum": {
"field": "msg_size"
}
}
}
}
}
}
}
}'
response:
{
"took" : 33,
"timed_out" : false,
"_shards" : {
"total" : 15,
"successful" : 10,
"skipped" : 0,
"failed" : 5,
"failures" : [
{
"shard" : 0,
"index" : "rollup_ndx-log-000001",
"node" : "3BzenQDBRqqQfPS6LtoMIw",
"reason" : {
"type" : "illegal_argument_exception",
"reason" : "Not all indices have rollup job"
}
}
]
},
"hits" : {
"total" : {
"value" : 16,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"message_numbers" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Success",
"doc_count" : 13,
"per_message" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Success",
"doc_count" : 13,
"sum_message" : {
"value" : 490.0
}
}
]
}
},
{
"key" : "Error",
"doc_count" : 3,
"per_message" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Error",
"doc_count" : 3,
"sum_message" : {
"value" : 90.0
}
}
]
}
}
]
}
}
}