I just stumbled upon sensu yesterday, and started to play with it. I am trying to figure out the proper flow of how I could implement a “auto correct” attempt, before sending an alert.
For example, if cron is not running on a server I would want to attempt to start cron before raising the event as a critical My first thought would be to have that in the handlers. I would have a check run the first handler to attempt to start the process, then set a refresh of 60 seconds, the second handler would check if cron was still down and then send an email. I don’t think this will work since handlers only run on the server. Would a better approach to have the “auto correct” attempt built into the check script itself? If the script was able to auto correct the event, I would still want a handler to send an email to update me that it was able to fix it. Do handlers only trigger when an event is in “critical”?