Sensu / PagerDuty integration


#1

Howdy folks,

Our pagerduty handler works, but it’s flooding the logs with this:

{“timestamp”:“2015-03-05T10:34:31.793284-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:"/etc/sensu/handlers/pagerduty.rb:34:in block in handle': undefined method’ for nil:NilClass (NoMethodError)\n"}

{“timestamp”:“2015-03-05T10:34:31.793520-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:"\tfrom /opt/sensu/embedded/lib/ruby/2.0.0/timeout.rb:66:in `timeout’\n"}

{“timestamp”:“2015-03-05T10:34:31.793643-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:"\tfrom /opt/sensu/embedded/lib/ruby/2.0.0/timeout.rb:97:in `timeout’\n"}

{“timestamp”:“2015-03-05T10:34:31.793739-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:"\tfrom /etc/sensu/handlers/pagerduty.rb:19:in `handle’\n"}

{“timestamp”:“2015-03-05T10:34:31.793831-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:"\tfrom /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-plugin-1.1.0/lib/sensu-handler.rb:55:in `block in class:Handler’\n"}

{“timestamp”:“2015-03-05T10:34:31.793939-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:“timed out while attempting to query the sensu api for a stash\n”}

{“timestamp”:“2015-03-05T10:34:31.794039-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:“timed out while attempting to query the sensu api for a stash\n”}

{“timestamp”:“2015-03-05T10:34:31.794126-0700”,“level”:“info”,“message”:“handler output”,“handler”:{“command”:"/etc/sensu/handlers/pagerduty.rb",“type”:“pipe”,“name”:“pagerduty”},“output”:“timed out while attempting to query the sensu api for a stash\n”}

Is there anything we can do to fix this? I can only imagine this getting a lot noisier as we add more servers, to the point of creating obscenely large log files. My Ruby knowledge is quite limited, and the fact that it works makes these errors confusing.


#2

Is sensu-api up and running?

If you install your sensu server on ubuntu, you can check it by running sudo service sensu-api status