clients always return status 0, but server recieves status 127

Sensu Classic (EOL)
1

I’ve searched sensu-client.log on the Sensu client machine, there’s NO status value is 127, all status are 0. But in the sensu-server.log on the Sensu server machine, the result received by server contains both status 0 and status 127. (See the “history” attribute in sensu-server.log)

Here is a piece of sensu-client.log:
{“timestamp”:“2015-03-11T08:47:39.481509+0800”,“level”:“info”,“message”:“received check request”,“check”:{“name”:“cron_check”,“issued”:1426034797,“command”:"/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf"}}

{“timestamp”:“2015-03-11T08:47:39.553249+0800”,“level”:“info”,“message”:“publishing check result”,“payload”:{“client”:“agent-01”,“check”:{“name”:“cron_check”,“issued”:1426034797,“command”:"/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf",“executed”:1426034859,“duration”:0.071,“output”:“CheckMySQL OK: Server version: 5.6.23\n”,“status”:0}}}

Here is a piece of sensu-server.log:
{“timestamp”:“2015-03-11T08:47:37.702340+0800”,“level”:“info”,“message”:“publishing check request”,“payload”:{“name”:“cron_check”,“issued”:1426034857,“command”:"/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf"},“subscribers”:[“webservers”]}

{“timestamp”:“2015-03-11T08:47:37.724937+0800”,“level”:“info”,“message”:“handling event”,“event”:{“id”:“1db13c85-c260-4b31-b116-e80cbc08409a”,“client”:{“name”:“agent-01”,“address”:“172.16.0.120”,“subscriptions”:[“all”,“webservers”],“version”:“0.16.0”,“timestamp”:1426034843},“check”:{“handlers”:[“print_timestamp”],“command”:"/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf",“interval”:10,“subscribers”:[“webservers”],“name”:“cron_check”,“issued”:1426034857,“executed”:1426034857,“duration”:0.008,“output”:“sh: 1: /etc/sensu/plugins/mysql-alive.rb: not found\n”,“status”:127,“history”:[“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”]},“occurrences”:1,“action”:“create”},“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“print_timestamp”}}

{“timestamp”:“2015-03-11T08:47:37.780722+0800”,“level”:“info”,“message”:“handling event”,“event”:{“id”:“5df36336-8804-45cd-be3e-3b261d1f3a98”,“client”:{“name”:“agent-01”,“address”:“172.16.0.120”,“subscriptions”:[“all”,“webservers”],“version”:“0.16.0”,“timestamp”:1426034843},“check”:{“handlers”:[“print_timestamp”],“command”:"/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf",“interval”:10,“subscribers”:[“webservers”],“name”:“cron_check”,“issued”:1426034857,“executed”:1426034919,“duration”:0.067,“output”:“CheckMySQL OK: Server version: 5.6.23\n”,“status”:0,“history”:[“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”]},“occurrences”:1,“action”:“resolve”},“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“print_timestamp”}}

{“timestamp”:“2015-03-11T08:47:38.451498+0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“print_timestamp”},“output”:“mail – sent alert for agent-01/cron_check to xxx@yyy.com\n”}

{“timestamp”:“2015-03-11T08:47:38.484025+0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:"/etc/sensu/handlers/mailer.rb",“name”:“print_timestamp”},“output”:“mail – sent alert for agent-01/cron_check to xxx@yyy.com\n”}

I have 2 question here:

  1. How could the Sensu server receive check output with status 127, while all the check outputs are status 0 on Sensu client?
  2. Why does the status 0 also triger a handler? (See line 3 in the sensu-server.log, the status is 0, but it also trigered a handler)
2

Solved it !!!
on the Sensu server, I also started sensu-client service, which subscribe to the same subcription as Sensu client, but don’t have the mysql-alive.rb script on it.
When I stop sensu-client service on Sensu server, it works well.

在 2015年3月11日星期三 UTC+8上午9:58:34，Timothy John写道：

···

I’ve searched sensu-client.log on the Sensu client machine, there’s NO status value is 127, all status are 0. But in the sensu-server.log on the Sensu server machine, the result received by server contains both status 0 and status 127. (See the “history” attribute in sensu-server.log)

Here is a piece of sensu-client.log:
{“timestamp”:“2015-03-11T08:47:39.481509+0800”,“level”:“info”,“message”:“received check request”,“check”:{“name”:“cron_check”,“issued”:1426034797,“command”:“/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf”}}

{“timestamp”:“2015-03-11T08:47:39.553249+0800”,“level”:“info”,“message”:“publishing check result”,“payload”:{“client”:“agent-01”,“check”:{“name”:“cron_check”,“issued”:1426034797,“command”:“/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf”,“executed”:1426034859,“duration”:0.071,“output”:“CheckMySQL OK: Server version: 5.6.23\n”,“status”:0}}}

Here is a piece of sensu-server.log:
{“timestamp”:“2015-03-11T08:47:37.702340+0800”,“level”:“info”,“message”:“publishing check request”,“payload”:{“name”:“cron_check”,“issued”:1426034857,“command”:“/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf”},“subscribers”:[“webservers”]}

{“timestamp”:“2015-03-11T08:47:37.724937+0800”,“level”:“info”,“message”:“handling event”,“event”:{“id”:“1db13c85-c260-4b31-b116-e80cbc08409a”,“client”:{“name”:“agent-01”,“address”:“172.16.0.120”,“subscriptions”:[“all”,“webservers”],“version”:“0.16.0”,“timestamp”:1426034843},“check”:{“handlers”:[“print_timestamp”],“command”:“/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf”,“interval”:10,“subscribers”:[“webservers”],“name”:“cron_check”,“issued”:1426034857,“executed”:1426034857,“duration”:0.008,“output”:“sh: 1: /etc/sensu/plugins/mysql-alive.rb: not found\n”,“status”:127,“history”:[“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”]},“occurrences”:1,“action”:“create”},“handler”:{“type”:“pipe”,“command”:“/etc/sensu/handlers/mailer.rb”,“name”:“print_timestamp”}}

{“timestamp”:“2015-03-11T08:47:37.780722+0800”,“level”:“info”,“message”:“handling event”,“event”:{“id”:“5df36336-8804-45cd-be3e-3b261d1f3a98”,“client”:{“name”:“agent-01”,“address”:“172.16.0.120”,“subscriptions”:[“all”,“webservers”],“version”:“0.16.0”,“timestamp”:1426034843},“check”:{“handlers”:[“print_timestamp”],“command”:“/etc/sensu/plugins/mysql-alive.rb -h localhost --ini /etc/sensu/my.conf”,“interval”:10,“subscribers”:[“webservers”],“name”:“cron_check”,“issued”:1426034857,“executed”:1426034919,“duration”:0.067,“output”:“CheckMySQL OK: Server version: 5.6.23\n”,“status”:0,“history”:[“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”,“127”,“0”]},“occurrences”:1,“action”:“resolve”},“handler”:{“type”:“pipe”,“command”:“/etc/sensu/handlers/mailer.rb”,“name”:“print_timestamp”}}

{“timestamp”:“2015-03-11T08:47:38.451498+0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:“/etc/sensu/handlers/mailer.rb”,“name”:“print_timestamp”},“output”:“mail – sent alert for agent-01/cron_check to xxx@yyy.com\n”}

{“timestamp”:“2015-03-11T08:47:38.484025+0800”,“level”:“info”,“message”:“handler output”,“handler”:{“type”:“pipe”,“command”:“/etc/sensu/handlers/mailer.rb”,“name”:“print_timestamp”},“output”:“mail – sent alert for agent-01/cron_check to xxx@yyy.com\n”}

I have 2 question here:

  1. How could the Sensu server receive check output with status 127, while all the check outputs are status 0 on Sensu client?
  2. Why does the status 0 also triger a handler? (See line 3 in the sensu-server.log, the status is 0, but it also trigered a handler)