Creating alerts when disk space exceeds 80 percent

OpenSearch
1

version : 2.5.0

I have a requirement where i need to send alerts to my teammates(microsoft teams or email). when the disk percent exceeds 80 . in our opensearch cluster i see alerting option where it has(per query monitor, per bucket monitor , per document monitor and per cluster monitor).

i would basically like to execuste _cat/allocation api in some sort of way through these monitors and define alerting. the closest option i could relate is per cluster monitor but it had pre defined set of APIs like(cluster health, node stats) none of them could give me the disk.percent details.

can someone guide what is the best approach to create the alert for above mentioned requirement?

2

Node stats api can return each node’s disk usage, that is in the fs.total section of the response as below:

"fs": {
        "timestamp": 1696948624891,
        "total": {
          "total_in_bytes": 62671097856,
          "free_in_bytes": 29389557760,
          "available_in_bytes": 26172841984,
          "cache_reserved_in_bytes": 0
        },

. If you want to send alert when one of the node’s disk usage exceeds 80%, you can use per cluster metrics monitor, select node stats API, and define a trigger like this:

for (entry in ctx.results[0].nodes.entrySet())
{
    if ((entry.getValue().fs.total.total_in_bytes -entry.getValue().fs.total.free_in_bytes)*100/entry.getValue().fs.total.total_in_bytes > 80) {
        return true;
}
}
return false;

, hope this helps.

3

Thanks a lot for this detailed explanation