Hi, is it possible to monitor if indices being ceated daily and logs flowing? we would like to be notified some sources stop sending logs. (it could be anomaly dedection which does automatically or we may add which index to watch over)
Hi. The short answer is yes.
The long answer is that there are two cases:
- Alert if ingestion slows down or stops for a while. You’d monitor the number of indexed documents via Indices Stats or Nodes Stats.
- Alert if indices aren’t rotated properly (e.g. if an index grew too large). In that case, you might alert on index size (from Indices Stats).
Typically, you’d use a monitoring tool like Sematext Monitoring to do that. I work for Sematext and contributed the OpenSearch integration, so I’m biased, but we can alert you on both thresholds or anomalies of these specific metrics (and much more). But of course any tool that can alert on these metrics should do.