sensu-api /event/resolve issue


#1

I installed Sensu on Debian Wheezy using the omnibus package. Things are working fine except for this one issue I am facing. I cannot resolve events using sensu-api. Is anyone else using the API to post updates? Can this work?

My goal is to allow Pagerduty to resolve Sensu events using Pagerduty webhooks. I am testing to see how the sensu-api works for resolving active events.

I am monitoring a cron process on my sensu client. I stopped the process on the client cFQDN and waited for the event to show in the sensu-server log:
(Note: cFQDN is my client and hFQDN is the sensu server.)

{“timestamp”:“2014-01-30T15:59:15.206641-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/pagerduty.rb",“name”:“pagerduty”},“output”:“pagerduty – Created incident – cFQDN/cron_check\n”}
{“timestamp”:“2014-01-30T15:59:15.868650-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“mailer”},“output”:“mail – sent alert for cFQDN/cron_check to sensu@email.info\n”}

I get the alert email and see the incident on Pagerduty:
Description: cFQDN : cron_check : CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/
Incident Key: cFQDN/cron_check

Now I want to manually resolve the event using the sensu-api.
I use this php with Interactive mode enabled:
php > $username = “user”;
php > $password = “pass”;
php > $data = array(“client” => “cFQDN”, “check” => “cron_check”);
php > $data_string = json_encode($data);
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_USERPWD, “$username:$password”);
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567”);
php > $result = curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);
php > $result = curl_exec($ch);
[{“output”:“CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/\n”,“status”:2,“issued”:1391128634,“handlers”:[“mailer”,“pagerduty”],“flapping”:false,“occurrences”:3,“client”:“cFQDN”,“check”:“cron_check”}]php > echo $result,"\n";
1
php > curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/event/resolve”);
php > $result = curl_exec($ch);
php > echo $result,"\n";
1
php > curl_close($ch);

So access through sensu-api is working. The login occurs and the /events get displayed. When the /event/resolve POST is attempted it does not resolve the event.

These are the results from sensu-api log:
{“timestamp”:“2014-01-30T16:03:21.118570-0800”,“level”:“info”,“message”:“GET /events”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“GET”,“request_uri”:"/events",“request_body”:""}
{“timestamp”:“2014-01-30T16:03:21.124943-0800”,“level”:“info”,“message”:“POST /event/resolve”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“POST”,“request_uri”:"/event/resolve",“request_body”:"{“client”:“cFQDN”,“check”:“cron_check”}"}


#2

Dan,

Change your post url to http://hFQDN:4567/resolve

If that doesn’t work I’ll test your code.

-Bryan

···

On Thu, Jan 30, 2014 at 7:38 PM, Dan Fischer ebay2007buys@gmail.com wrote:

I installed Sensu on Debian Wheezy using the omnibus package. Things are working fine except for this one issue I am facing. I cannot resolve events using sensu-api. Is anyone else using the API to post updates? Can this work?

My goal is to allow Pagerduty to resolve Sensu events using Pagerduty webhooks. I am testing to see how the sensu-api works for resolving active events.

I am monitoring a cron process on my sensu client. I stopped the process on the client cFQDN and waited for the event to show in the sensu-server log:

(Note: cFQDN is my client and hFQDN is the sensu server.)

{“timestamp”:“2014-01-30T15:59:15.206641-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/pagerduty.rb",“name”:“pagerduty”},“output”:“pagerduty – Created incident – cFQDN/cron_check\n”}

{“timestamp”:“2014-01-30T15:59:15.868650-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“mailer”},“output”:“mail – sent alert for cFQDN/cron_check to sensu@email.info\n”}

I get the alert email and see the incident on Pagerduty:
Description: cFQDN : cron_check : CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/
Incident Key: cFQDN/cron_check

Now I want to manually resolve the event using the sensu-api.
I use this php with Interactive mode enabled:
php > $username = “user”;
php > $password = “pass”;

php > $data = array(“client” => “cFQDN”, “check” => “cron_check”);
php > $data_string = json_encode($data);
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_USERPWD, “$username:$password”);

php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567”);
php > $result = curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);

php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);
php > $result = curl_exec($ch);
[{“output”:“CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/\n”,“status”:2,“issued”:1391128634,“handlers”:[“mailer”,“pagerduty”],“flapping”:false,“occurrences”:3,“client”:“cFQDN”,“check”:“cron_check”}]php > echo $result,"\n";

1
php > curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/event/resolve”);
php > $result = curl_exec($ch);

php > echo $result,"\n";
1
php > curl_close($ch);

So access through sensu-api is working. The login occurs and the /events get displayed. When the /event/resolve POST is attempted it does not resolve the event.

These are the results from sensu-api log:
{“timestamp”:“2014-01-30T16:03:21.118570-0800”,“level”:“info”,“message”:“GET /events”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“GET”,“request_uri”:"/events",“request_body”:""}

{“timestamp”:“2014-01-30T16:03:21.124943-0800”,“level”:“info”,“message”:“POST /event/resolve”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“POST”,“request_uri”:"/event/resolve",“request_body”:"{“client”:“cFQDN”,“check”:“cron_check”}"}


#3

Dan,

Change your post url to http://hFQDN:4567/resolve

If that doesn’t work I’ll test your code.

-Bryan

I installed Sensu on Debian Wheezy using the omnibus package. Things are working fine except for this one issue I am facing. I cannot resolve events using sensu-api. Is anyone else using the API to post updates? Can this work?

My goal is to allow Pagerduty to resolve Sensu events using Pagerduty webhooks. I am testing to see how the sensu-api works for resolving active events.

I am monitoring a cron process on my sensu client. I stopped the process on the client cFQDN and waited for the event to show in the sensu-server log:

(Note: cFQDN is my client and hFQDN is the sensu server.)

{“timestamp”:“2014-01-30T15:59:15.206641-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/pagerduty.rb",“name”:“pagerduty”},“output”:“pagerduty – Created incident – cFQDN/cron_check\n”}

{“timestamp”:“2014-01-30T15:59:15.868650-0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“mailer”},“output”:“mail – sent alert for cFQDN/cron_check to se…@email.info\n”}

I get the alert email and see the incident on Pagerduty:
Description: cFQDN : cron_check : CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/
Incident Key: cFQDN/cron_check

Now I want to manually resolve the event using the sensu-api.
I use this php with Interactive mode enabled:
php > $username = “user”;
php > $password = “pass”;

php > $data = array(“client” => “cFQDN”, “check” => “cron_check”);
php > $data_string = json_encode($data);
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_USERPWD, “$username:$password”);

php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567”);
php > $result = curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);

php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/events”);
php > $result = curl_exec($ch);
[{“output”:“CheckProcs CRITICAL: Found 0 matching processes; cmd /cron/\n”,“status”:2,“issued”:1391128634,“handlers”:[“mailer”,“pagerduty”],“flapping”:false,“occurrences”:3,“client”:“cFQDN”,“check”:“cron_check”}]php > echo $result,"\n";

1
php > curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/event/resolve”);
php > $result = curl_exec($ch);

php > echo $result,"\n";
1
php > curl_close($ch);

So access through sensu-api is working. The login occurs and the /events get displayed. When the /event/resolve POST is attempted it does not resolve the event.

These are the results from sensu-api log:
{“timestamp”:“2014-01-30T16:03:21.118570-0800”,“level”:“info”,“message”:“GET /events”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“GET”,“request_uri”:"/events",“request_body”:""}

{“timestamp”:“2014-01-30T16:03:21.124943-0800”,“level”:“info”,“message”:“POST /event/resolve”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“POST”,“request_uri”:"/event/resolve",“request_body”:"{“client”:“cFQDN”,“check”:“cron_check”}"}

Thanks Bryan. Changing the POST to /resolve:
php > curl_setopt($ch, CURLOPT_URL, “http://hFQDN:4567/resolve”);
php > $result = curl_exec($ch);
{“issued”:1391136306}

Seems this fixed it - results in sensu-api log:
{“timestamp”:“2014-01-30T18:38:41.471124-0800”,“level”:“info”,“message”:“POST /resolve”,“remote_address”:“xxx.xxx.xx.xxx”,“user_agent”:null,“request_method”:“POST”,“request_uri”:"/resolve",“request_body”:"{“client”:“cFQDN”,“check”:“cron_check”}"}
{“timestamp”:“2014-01-30T18:38:41.471858-0800”,“level”:“info”,“message”:“publishing check result”,“payload”:{“client”:“cFQDN”,“check”:{“name”:“cron_check”,“output”:“Resolving on request of the API”,“status”:0,“issued”:1391135921,“handlers”:[“mailer”,“pagerduty”],“force_resolve”:true}}}

···

On Thu, Jan 30, 2014 at 7:38 PM, Dan Fischer ebay20...@gmail.com wrote:

On Thursday, January 30, 2014 7:07:51 PM UTC-7, agent462 wrote:


#4