Hello,
We got a lot of this ones in our sensu-server log:
{“timestamp”:“2016-12-08T17:31:33.651443+0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/handler-opsgenie.rb",“severities”:[“ok”,“warning”,“critical”,“unknown”],“type”:“pipe”,“name”:“opsgenie”},“output”:[“timed out while attempting to query the sensu api for a stash\n”,“opsgenie – failed to create incident – xxxx/keepalive\n”]}
This result in a lot of false alerts where there is actually a check stashed but sensu times out on the API to check for it and create an event.
In a loop I can query the sensu API and get a response in some ms, but sometimes it hangs and takes some seconds, this is where it runs in the 5 sec timeout from the sensu-handler.rb.
I increased the sensu API threads but it hadn’t any effect.
Any help where to look highly appreciated.