Working on Transform Job for aggregating data to raw to hourly and putting data_selection_query to get data older than 1 hour

Community Jobs
1

using following transform job
“transform”: {
“enabled”: true,
“continuous”: true,
“source_index”: “index-raw-*”,
“target_index”: “index-intermediate-hourly”,
“description”: “Hourly rollup ”,
“page_size”: 500,
“schedule”: { “interval”: { “period”: 10, “unit”: “Minutes” } },
“data_selection_query”: {
“range”: {
“@timestamp”: {
“lt”: “now-1h”
}
}
},
….}
and indexing is working but after some processing some page it is giving error

image
image1550×224 13.4 KB

2

@gourabh09 Could you check your OpenSearch logs for corresponding errors?

3

@pablo reducing page size to 200 works

1 Like