I’m wondering if it’s possible to do edge-triggered checks in Sensu.
All the checks out there seems to be level-triggered meaning that they look at the current value and alert whenever the value breaches a limit regardless of the previous value. Edge-triggered checks are sometimes useful. A motivating example is Ultra DMA CRC Error Count in SMART. This counter will never be reset even if the problem goes away by e.g. replacing a
failing cable. So a level-triggered check (e.g. check-smart-status.rb) will stop working once we get a increase in the counter.
One possible way is to record the counter on Graphite and use check-graphite-data.rb with taking nonNegativeDrivative of the series. It feels like an involved solution though.
Are there any other simpler ways to do edge-triggered checks?