stop console output

#1

Brand new to sensu and using linux for only a few years… I’ve been given the monitoring duties at my job and have been using Icinga for a year or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the structure and basic setup. It will be interesting to see how it develops as I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,

Chris.

#2

Are you not using the package and service scripts? They handle the logger.

···

On Jun 18, 2014 2:42 PM, “Chris Jefferies” chris@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been given the monitoring duties at my job and have been using Icinga for a year or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the structure and basic setup. It will be interesting to see how it develops as I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,

Chris.

#3

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet. Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the server could be running without ruby. On the client, I had to track down a config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you mean. I do see log files (at /var/log/sensu/), so apparently those are being handled. I’m just not sure where the hook is located to turn off the annoying flood of log data that streams into the console from sense. It looks like debug info.

Thanks for your help,

Chris.

···

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been given the monitoring duties at my job and have been using Icinga for a year or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the structure and basic setup. It will be interesting to see how it develops as I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,

Chris.

#4

Are you literally running /opt/sensu/bin/sensu-server on the command line?
Usually using the init scripts /etc/init.d/sensu-server calls that, in
order to properly handle the ruby part, configuration, permissions,
user, and logs.

···

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies <chris@freeranger.com> wrote:

Sorry, portertech,

I don't understand what you mean by package and service scripts... yet.
Hopefully I'll know soon.

I installed via the omnibus system and it took a while to understand how the
server could be running without ruby. On the client, I had to track down a
config item that sets EMBEDDED_RUBY=true. So I've got it mostly running.

I have not loaded any specific ruby or gem packages, if that's what you
mean. I do see log files (at /var/log/sensu/), so apparently those are
being handled. I'm just not sure where the hook is located to turn off the
annoying flood of log data that streams into the console from sense. It
looks like debug info.

Thanks for your help,
Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, "Chris Jefferies" <ch...@freeranger.com> wrote:

Brand new to sensu and using linux for only a few years... I've been
given the monitoring duties at my job and have been using Icinga for a year
or so.

I've got sensu running on a couple of hosts and so far I'm liking the
structure and basic setup. It will be interesting to see how it develops as
I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how
to turn off the console output from sensu.

I'm getting a lot of console output that looks like this:

{"timestamp":"2014-06-18T21:38:24.042751+0000","level":"info","message":"determining
stale clients"}

{"timestamp":"2014-06-18T21:38:24.090478+0000","level":"info","message":"pruning
aggregations"}

I'm getting these entries on both the server and the client.

Any tips appreciated,
Chris.

#5

I’m using the startup approach described in the online guide at: http://sensuapp.org/docs/0.12/guide

/etc/init.d/sensu-server restart

/etc/init.d/sensu-client start

whenever I make config changes.

As soon as I start I see something like this:

Starting sensu-server{“timestamp”:“2014-06-19T16:09:47.183208+0000”,“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

{“timestamp”:“2014-06-19T16:09:47.183736+0000”,“level”:“error”,“message”:“config file does not exist or is not readable”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.183797+0000”,“level”:“warn”,“message”:“ignoring config file”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.185148+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_cron.json",“changes”:{“checks”:{“cron_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",“interval”:60,“subscribers”:[“webservers”,“general”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185412+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:"",“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.185666+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_haproxy.json",“changes”:{“checks”:{“haproxy_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p haproxy -C 1 ",“interval”:60,“subscribers”:[“appliance”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185852+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:"",“address”:“10.0.2.19”,“subscriptions”:[“all”,“SensuServer”]}]}}

{“timestamp”:“2014-06-19T16:09:47.186266+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186571+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/dashboard.json",“changes”:{“dashboard”:[null,{“port”:8080,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186760+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}

{“timestamp”:“2014-06-19T16:09:47.240382+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“mutator”,“name”:“only_check_output”,“description”:“returns check output”}

{“timestamp”:“2014-06-19T16:09:47.259583+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“handler”,“name”:“debug”,“description”:“outputs json event data”}

{“timestamp”:“2014-06-19T16:09:47.512081+0000”,“level”:“info”,“message”:“i am now the master”}

It looks like log output but it is streaming onto the console. There’s so much output that it’s hard to type at the command line. Sometimes I have to open another terminal window to stop the service.

Crazy. I’ll bet it’s something simple.

Thanks for your help,

Chris.

···

On Thursday, June 19, 2014 8:42:03 AM UTC-7, Kyle Anderson wrote:

Are you literally running /opt/sensu/bin/sensu-server on the command line?
Usually using the init scripts /etc/init.d/sensu-server calls that, in
order to properly handle the ruby part, configuration, permissions,
user, and logs.

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies ch...@freeranger.com wrote:

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet.
Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the
server could be running without ruby. On the client, I had to track down a
config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you
mean. I do see log files (at /var/log/sensu/), so apparently those are
being handled. I’m just not sure where the hook is located to turn off the
annoying flood of log data that streams into the console from sense. It
looks like debug info.

Thanks for your help,
Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been
given the monitoring duties at my job and have been using Icinga for a year
or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the
structure and basic setup. It will be interesting to see how it develops as
I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how
to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining
stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning
aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,
Chris.

#6

Look at the first message of your log output:

Starting sensu-server{“timestamp”:"
2014-06-19T16:09:47.183208+0000",“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

You need to investigate why the log file is not writable. Is there an ownership problem for the directory? It needs to match the owner of the user running the sensu job (usually user ‘sensu’).

With the log file being unwritable, it’s falling back to stdout.

Good luck!

Mojo

···

On Thu, Jun 19, 2014 at 9:15 AM, Chris Jefferies chris@freeranger.com wrote:

I’m using the startup approach described in the online guide at: http://sensuapp.org/docs/0.12/guide

/etc/init.d/sensu-server restart

/etc/init.d/sensu-client start

whenever I make config changes.

As soon as I start I see something like this:

Starting sensu-server{“timestamp”:“2014-06-19T16:09:47.183208+0000”,“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

{“timestamp”:“2014-06-19T16:09:47.183736+0000”,“level”:“error”,“message”:“config file does not exist or is not readable”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.183797+0000”,“level”:“warn”,“message”:“ignoring config file”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.185148+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_cron.json",“changes”:{“checks”:{“cron_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",“interval”:60,“subscribers”:[“webservers”,“general”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185412+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:"",“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.185666+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_haproxy.json",“changes”:{“checks”:{“haproxy_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p haproxy -C 1 ",“interval”:60,“subscribers”:[“appliance”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185852+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:"",“address”:“10.0.2.19”,“subscriptions”:[“all”,“SensuServer”]}]}}

{“timestamp”:“2014-06-19T16:09:47.186266+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186571+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/dashboard.json",“changes”:{“dashboard”:[null,{“port”:8080,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186760+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}

{“timestamp”:“2014-06-19T16:09:47.240382+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“mutator”,“name”:“only_check_output”,“description”:“returns check output”}

{“timestamp”:“2014-06-19T16:09:47.259583+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“handler”,“name”:“debug”,“description”:“outputs json event data”}

{“timestamp”:“2014-06-19T16:09:47.512081+0000”,“level”:“info”,“message”:“i am now the master”}

It looks like log output but it is streaming onto the console. There’s so much output that it’s hard to type at the command line. Sometimes I have to open another terminal window to stop the service.

Crazy. I’ll bet it’s something simple.

Thanks for your help,

Chris.

On Thursday, June 19, 2014 8:42:03 AM UTC-7, Kyle Anderson wrote:

Are you literally running /opt/sensu/bin/sensu-server on the command line?
Usually using the init scripts /etc/init.d/sensu-server calls that, in
order to properly handle the ruby part, configuration, permissions,
user, and logs.

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies ch...@freeranger.com wrote:

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet.
Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the
server could be running without ruby. On the client, I had to track down a
config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you
mean. I do see log files (at /var/log/sensu/), so apparently those are
being handled. I’m just not sure where the hook is located to turn off the
annoying flood of log data that streams into the console from sense. It
looks like debug info.

Thanks for your help,
Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been
given the monitoring duties at my job and have been using Icinga for a year
or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the
structure and basic setup. It will be interesting to see how it develops as
I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how
to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining
stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning
aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,
Chris.

#7

Sorry, perhaps I should mention rhel 6, Omnibus install, straight ahead install from Guide (worked without a hitch), converting my client side nrpe commands…

···

On Thursday, June 19, 2014 9:15:44 AM UTC-7, Chris Jefferies wrote:

I’m using the startup approach described in the online guide at: http://sensuapp.org/docs/0.12/guide

/etc/init.d/sensu-server restart

/etc/init.d/sensu-client start

whenever I make config changes.

As soon as I start I see something like this:

Starting sensu-server{“timestamp”:“2014-06-19T16:09:47.183208+0000”,“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

{“timestamp”:“2014-06-19T16:09:47.183736+0000”,“level”:“error”,“message”:“config file does not exist or is not readable”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.183797+0000”,“level”:“warn”,“message”:“ignoring config file”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.185148+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_cron.json",“changes”:{“checks”:{“cron_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",“interval”:60,“subscribers”:[“webservers”,“general”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185412+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:"",“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.185666+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_haproxy.json",“changes”:{“checks”:{“haproxy_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p haproxy -C 1 ",“interval”:60,“subscribers”:[“appliance”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185852+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:"",“address”:“10.0.2.19”,“subscriptions”:[“all”,“SensuServer”]}]}}

{“timestamp”:“2014-06-19T16:09:47.186266+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186571+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/dashboard.json",“changes”:{“dashboard”:[null,{“port”:8080,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186760+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}

{“timestamp”:“2014-06-19T16:09:47.240382+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“mutator”,“name”:“only_check_output”,“description”:“returns check output”}

{“timestamp”:“2014-06-19T16:09:47.259583+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“handler”,“name”:“debug”,“description”:“outputs json event data”}

{“timestamp”:“2014-06-19T16:09:47.512081+0000”,“level”:“info”,“message”:“i am now the master”}

It looks like log output but it is streaming onto the console. There’s so much output that it’s hard to type at the command line. Sometimes I have to open another terminal window to stop the service.

Crazy. I’ll bet it’s something simple.

Thanks for your help,

Chris.

On Thursday, June 19, 2014 8:42:03 AM UTC-7, Kyle Anderson wrote:

Are you literally running /opt/sensu/bin/sensu-server on the command line?
Usually using the init scripts /etc/init.d/sensu-server calls that, in
order to properly handle the ruby part, configuration, permissions,
user, and logs.

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies ch...@freeranger.com wrote:

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet.
Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the
server could be running without ruby. On the client, I had to track down a
config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you
mean. I do see log files (at /var/log/sensu/), so apparently those are
being handled. I’m just not sure where the hook is located to turn off the
annoying flood of log data that streams into the console from sense. It
looks like debug info.

Thanks for your help,
Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been
given the monitoring duties at my job and have been using Icinga for a year
or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the
structure and basic setup. It will be interesting to see how it develops as
I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how
to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:“determining
stale clients”}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:“pruning
aggregations”}

I’m getting these entries on both the server and the client.

Any tips appreciated,
Chris.

#8

Good eye, Mojo. That gives me something to look at. Thanks.

···

On Thursday, June 19, 2014 9:19:05 AM UTC-7, Morris Jones wrote:

Look at the first message of your log output:

Starting sensu-server{“timestamp”:"
2014-06-19T16:09:47.183208+0000",“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

You need to investigate why the log file is not writable. Is there an ownership problem for the directory? It needs to match the owner of the user running the sensu job (usually user ‘sensu’).

With the log file being unwritable, it’s falling back to stdout.

Good luck!

Mojo

On Thu, Jun 19, 2014 at 9:15 AM, Chris Jefferies ch...@freeranger.com wrote:

I’m using the startup approach described in the online guide at: http://sensuapp.org/docs/0.12/guide

/etc/init.d/sensu-server restart

/etc/init.d/sensu-client start

whenever I make config changes.

As soon as I start I see something like this:

Starting sensu-server{“timestamp”:“2014-06-19T16:09:47.183208+0000”,“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

{“timestamp”:“2014-06-19T16:09:47.183736+0000”,“level”:“error”,“message”:“config file does not exist or is not readable”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.183797+0000”,“level”:“warn”,“message”:“ignoring config file”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.185148+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_cron.json",“changes”:{“checks”:{“cron_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",“interval”:60,“subscribers”:[“webservers”,“general”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185412+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:"",“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.185666+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_haproxy.json",“changes”:{“checks”:{“haproxy_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p haproxy -C 1 ",“interval”:60,“subscribers”:[“appliance”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185852+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:"",“address”:“10.0.2.19”,“subscriptions”:[“all”,“SensuServer”]}]}}

{“timestamp”:“2014-06-19T16:09:47.186266+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186571+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/dashboard.json",“changes”:{“dashboard”:[null,{“port”:8080,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186760+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}

{“timestamp”:“2014-06-19T16:09:47.240382+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“mutator”,“name”:“only_check_output”,“description”:“returns check output”}

{“timestamp”:“2014-06-19T16:09:47.259583+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“handler”,“name”:“debug”,“description”:“outputs json event data”}

{“timestamp”:“2014-06-19T16:09:47.512081+0000”,“level”:“info”,“message”:“i am now the master”}

It looks like log output but it is streaming onto the console. There’s so much output that it’s hard to type at the command line. Sometimes I have to open another terminal window to stop the service.

Crazy. I’ll bet it’s something simple.

Thanks for your help,

Chris.

On Thursday, June 19, 2014 8:42:03 AM UTC-7, Kyle Anderson wrote:

Are you literally running /opt/sensu/bin/sensu-server on the command line?

Usually using the init scripts /etc/init.d/sensu-server calls that, in

order to properly handle the ruby part, configuration, permissions,

user, and logs.

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies ch...@freeranger.com wrote:

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet.

Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the

server could be running without ruby. On the client, I had to track down a

config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you

mean. I do see log files (at /var/log/sensu/), so apparently those are

being handled. I’m just not sure where the hook is located to turn off the

annoying flood of log data that streams into the console from sense. It

looks like debug info.

Thanks for your help,

Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been

given the monitoring duties at my job and have been using Icinga for a year

or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the

structure and basic setup. It will be interesting to see how it develops as

I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how

to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:"pruning

aggregations"}

I’m getting these entries on both the server and the client.

Any tips appreciated,

Chris.

#9

woohoo…

from the /var/log/ directory I ran this:

chown -R sensu:sensu sensu

all quiet now.

Thanks so much.

···

On Thursday, June 19, 2014 9:19:05 AM UTC-7, Morris Jones wrote:

Look at the first message of your log output:

Starting sensu-server{“timestamp”:"
2014-06-19T16:09:47.183208+0000",“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

You need to investigate why the log file is not writable. Is there an ownership problem for the directory? It needs to match the owner of the user running the sensu job (usually user ‘sensu’).

With the log file being unwritable, it’s falling back to stdout.

Good luck!

Mojo

On Thu, Jun 19, 2014 at 9:15 AM, Chris Jefferies ch...@freeranger.com wrote:

I’m using the startup approach described in the online guide at: http://sensuapp.org/docs/0.12/guide

/etc/init.d/sensu-server restart

/etc/init.d/sensu-client start

whenever I make config changes.

As soon as I start I see something like this:

Starting sensu-server{“timestamp”:“2014-06-19T16:09:47.183208+0000”,“level”:“error”,“message”:“log file is not writable”,“log_file”:"/var/log/sensu/sensu-server.log"}

{“timestamp”:“2014-06-19T16:09:47.183736+0000”,“level”:“error”,“message”:“config file does not exist or is not readable”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.183797+0000”,“level”:“warn”,“message”:“ignoring config file”,“config_file”:"/etc/sensu/config.json"}

{“timestamp”:“2014-06-19T16:09:47.185148+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_cron.json",“changes”:{“checks”:{“cron_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",“interval”:60,“subscribers”:[“webservers”,“general”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185412+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:"",“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.185666+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/check_haproxy.json",“changes”:{“checks”:{“haproxy_check”:[null,{“handlers”:[“default”],“command”:"/etc/sensu/plugins/check-procs.rb -p haproxy -C 1 ",“interval”:60,“subscribers”:[“appliance”]}]}}}

{“timestamp”:“2014-06-19T16:09:47.185852+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:"",“address”:“10.0.2.19”,“subscriptions”:[“all”,“SensuServer”]}]}}

{“timestamp”:“2014-06-19T16:09:47.186266+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186571+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/dashboard.json",“changes”:{“dashboard”:[null,{“port”:8080,“user”:“admin”,“password”:“REDACTED”}]}}

{“timestamp”:“2014-06-19T16:09:47.186760+0000”,“level”:“warn”,“message”:“config file applied changes”,“config_file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}

{“timestamp”:“2014-06-19T16:09:47.240382+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“mutator”,“name”:“only_check_output”,“description”:“returns check output”}

{“timestamp”:“2014-06-19T16:09:47.259583+0000”,“level”:“info”,“message”:“loaded extension”,“type”:“handler”,“name”:“debug”,“description”:“outputs json event data”}

{“timestamp”:“2014-06-19T16:09:47.512081+0000”,“level”:“info”,“message”:“i am now the master”}

It looks like log output but it is streaming onto the console. There’s so much output that it’s hard to type at the command line. Sometimes I have to open another terminal window to stop the service.

Crazy. I’ll bet it’s something simple.

Thanks for your help,

Chris.

On Thursday, June 19, 2014 8:42:03 AM UTC-7, Kyle Anderson wrote:

Are you literally running /opt/sensu/bin/sensu-server on the command line?

Usually using the init scripts /etc/init.d/sensu-server calls that, in

order to properly handle the ruby part, configuration, permissions,

user, and logs.

On Wed, Jun 18, 2014 at 5:34 PM, Chris Jefferies ch...@freeranger.com wrote:

Sorry, portertech,

I don’t understand what you mean by package and service scripts… yet.

Hopefully I’ll know soon.

I installed via the omnibus system and it took a while to understand how the

server could be running without ruby. On the client, I had to track down a

config item that sets EMBEDDED_RUBY=true. So I’ve got it mostly running.

I have not loaded any specific ruby or gem packages, if that’s what you

mean. I do see log files (at /var/log/sensu/), so apparently those are

being handled. I’m just not sure where the hook is located to turn off the

annoying flood of log data that streams into the console from sense. It

looks like debug info.

Thanks for your help,

Chris.

On Wednesday, June 18, 2014 2:46:56 PM UTC-7, portertech wrote:

Are you not using the package and service scripts? They handle the logger.

On Jun 18, 2014 2:42 PM, “Chris Jefferies” ch...@freeranger.com wrote:

Brand new to sensu and using linux for only a few years… I’ve been

given the monitoring duties at my job and have been using Icinga for a year

or so.

I’ve got sensu running on a couple of hosts and so far I’m liking the

structure and basic setup. It will be interesting to see how it develops as

I add more checks and some handlers.

First problem for which I cannot find an answer on google searches is how

to turn off the console output from sensu.

I’m getting a lot of console output that looks like this:

{“timestamp”:“2014-06-18T21:38:24.042751+0000”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2014-06-18T21:38:24.090478+0000”,“level”:“info”,“message”:"pruning

aggregations"}

I’m getting these entries on both the server and the client.

Any tips appreciated,

Chris.