Email notification every 10 min when failing

Sensu Go
,
1

Hello,
I’m trying to create a Filter than can send me an email notification when a check fails and every 10 min while keep failing.

So far if I use this:

"event.check.occurrences == 1"

I get a notification when the status changes from OK to Fail and back to OK. So, I guess that part is ok.

If I try to use:

"event.check.status == 1 || event.check.status == 2"

I got a notification every minute which is not what I want.

I even try this:

"event.check.status == 1 || event.check.status == 2 || event.check.occurrences % (300 / event.check.interval) == 0"

But that game me notifications for all Passing and Failing every 300 sec which is not what I want.

How can I add an expression to only send a notification is the check fails every 10 min?

Thanks.

2

check out the fatigue filter asset on bonsai: sensu-go-fatigue-check-filter versions

should fit your use pretty well

3

Hello,
Thanks for the link. Actually from that site is where I got:

"event.check.occurrences == 1 || event.check.occurrences % (3600 / event.check.interval)"

But that sends a notification for all checks every hour even is the status is OK which is not what I want.
I guess my question would be how to send a notification every 10 min. I guess that’s the part I’m missing.

4

I’m not sure you read that part of the document correctly, that was an example of a filter that is not ideal for the case… when you use this filter correctly, you can just add some annotations to accomplish your goal. go back and give the link another readthrough and you will understand better,

5

I think what you’re missing is the built-in is_incident filter. If used together with an occurrence filter you’ll achieve your desired outcome.

https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/filters/#built-in-filter-is_incident

I hope this helps!!