Sensu Client not sending Metrics data to the Server


#1

Hi,
We are facing a problem of not getting any check “metrics” response from the client while following the tutorial at link Adding a Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:“determining stale clients”}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:“publishing check request”,“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:“publishing check result”,“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:“Dev.load_avg.one 0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen 0.22 1425079537\n\n”,“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#2

It looks like there is no handler configured with the name of "default"
http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

···

On Fri, Feb 27, 2015 at 3:28 PM, <max2subscribe@gmail.com> wrote:

Hi,
           We are facing a problem of not getting any check "metrics"
response from the client while following the tutorial at link Adding a
Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
handler","handler_name":"default"}
{"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
stale clients"}
{"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
handler","handler_name":"default"}
{"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
handler","handler_name":"default"}
{"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
handler","handler_name":"default"}
{"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
check
request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}

Client > tail -f /var/log/sensu/sensu-client.log

{"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
check
result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen
0.22 1425079537\n\n","status":0}}}

What are the configs that we are missing ?

Thanks:-


#3

Hi,

I do have same issue where log states unknown handler email, may I know the need of having file.rb? I do not have this file in /etc/sensu/handlers/, do I need to install anything or add in /etc/sensu/conf.d/ to get started in generating alerts from server?

Please advise me…

···

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#4

Yes, you do need to install/deploy the handler script (mailer.rb, etc.
Usually in /etc/sensu/handlers/).
Additionally, you must add configuration for that handler
(/etc/sensu/conf.d/..) to configure that handler to perform actions on
events (to get alerts)
Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers
http://sensuapp.org/docs/0.17/handlers

···

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle <srini5884@gmail.com> wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the
need of having file.rb? I do not have this file in /etc/sensu/handlers/, do
I need to install anything or add in /etc/sensu/conf.d/ to get started in
generating alerts from server?

Please advise me...

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of "default"
http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> wrote:
> Hi,
> We are facing a problem of not getting any check "metrics"
> response from the client while following the tutorial at link Adding a
> Metric.
>
> Following are the outputs :-
>
> Server > tail -f /var/log/sensu/sensu-server.log
>
>
> {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
> handler","handler_name":"default"}
>
> {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
> stale clients"}
>
> {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
> handler","handler_name":"default"}
>
> {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
> handler","handler_name":"default"}
>
> {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
> handler","handler_name":"default"}
>
> {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
> check
>
> request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>
>
> Client > tail -f /var/log/sensu/sensu-client.log
>
>
> {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
> check
>
> result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
> 0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen
> 0.22 1425079537\n\n","status":0}}}
>
>
> What are the configs that we are missing ?
>
> Thanks:-
>


#5

In the API and in the dashboard (uchiwa) the last 20 status codes are
available in the "history" section of an event

http://sensuapp.org/docs/latest/events

Note that because sensu is not a "green light dashboard", this event
history is only user-visible when the check is failing, not on checks
that are green.
In that sense they are not really "stored" (in redis) unless the check
is failing. Otherwise they are reported by the client to the handler
as they come. (as event data)

···

On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle <srini5884@gmail.com> wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set up?
Can we get history with in server?

Just trying to where do all the results gets stored before handling to
handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson <kyle@xkyle.com> wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.
Usually in /etc/sensu/handlers/).
Additionally, you must add configuration for that handler
(/etc/sensu/conf.d/..) to configure that handler to perform actions on
events (to get alerts)
Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers
http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle <srini5884@gmail.com> >> wrote:
> Hi,
>
> I do have same issue where log states unknown handler email, may I know
> the
> need of having file.rb? I do not have this file in /etc/sensu/handlers/,
> do
> I need to install anything or add in /etc/sensu/conf.d/ to get started
> in
> generating alerts from server?
>
> Please advise me...
>
> On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:
>>
>> It looks like there is no handler configured with the name of "default"
>> http://sensuapp.org/docs/0.16/handlers
>>
>> Good handlers for metrics are things like graphite, etc?
>>
>> On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> wrote:
>> > Hi,
>> > We are facing a problem of not getting any check "metrics"
>> > response from the client while following the tutorial at link Adding
>> > a
>> > Metric.
>> >
>> > Following are the outputs :-
>> >
>> > Server > tail -f /var/log/sensu/sensu-server.log
>> >
>> >
>> >
>> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>> > handler","handler_name":"default"}
>> >
>> >
>> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>> > stale clients"}
>> >
>> >
>> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>> > handler","handler_name":"default"}
>> >
>> >
>> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>> > handler","handler_name":"default"}
>> >
>> >
>> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>> > handler","handler_name":"default"}
>> >
>> >
>> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>> > check
>> >
>> >
>> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>> >
>> >
>> > Client > tail -f /var/log/sensu/sensu-client.log
>> >
>> >
>> >
>> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>> > check
>> >
>> >
>> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>> > 0.13 1425079537\nDev.load_avg.five 0.12
>> > 1425079537\nDev.load_avg.fifteen
>> > 0.22 1425079537\n\n","status":0}}}
>> >
>> >
>> > What are the configs that we are missing ?
>> >
>> > Thanks:-
>> >


#6

You could use a graphite extension to store event history.
(I'm still waiting for some sort of cool sensudb thing to be invented
for historical analysis?)

Most of the time when I need to debug a hanlder I use the sensu-server
(sometimes in combination with the sensu-client) logs to try to figure
out why a handler or event didn't do what I thought it should do.

I guess in that sense the history is stored, just in log form (not
redis and not api)

···

On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle <srini5884@gmail.com> wrote:

Thanks Kyle,

In this case I may need to use graphite to store event history for certain
period of time for debugging whenever required.

Correct me if am wrong

On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson <kyle@xkyle.com> wrote:

In the API and in the dashboard (uchiwa) the last 20 status codes are
available in the "history" section of an event
http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg
http://sensuapp.org/docs/latest/events

Note that because sensu is not a "green light dashboard", this event
history is only user-visible when the check is failing, not on checks
that are green.
In that sense they are not really "stored" (in redis) unless the check
is failing. Otherwise they are reported by the client to the handler
as they come. (as event data)

On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle <srini5884@gmail.com> >> wrote:
> Thanks Kyle,
>
> May I know where can we get history of events in sensu monitoring set
> up?
> Can we get history with in server?
>
> Just trying to where do all the results gets stored before handling to
> handler.
>
> On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson <kyle@xkyle.com> wrote:
>>
>> Yes, you do need to install/deploy the handler script (mailer.rb, etc.
>> Usually in /etc/sensu/handlers/).
>> Additionally, you must add configuration for that handler
>> (/etc/sensu/conf.d/..) to configure that handler to perform actions on
>> events (to get alerts)
>> Read more about how to install and configure handlers at the docs page:
>>
>> http://sensuapp.org/docs/0.17/getting-started-with-handlers
>> http://sensuapp.org/docs/0.17/handlers
>>
>> On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle <srini5884@gmail.com> >> >> wrote:
>> > Hi,
>> >
>> > I do have same issue where log states unknown handler email, may I
>> > know
>> > the
>> > need of having file.rb? I do not have this file in
>> > /etc/sensu/handlers/,
>> > do
>> > I need to install anything or add in /etc/sensu/conf.d/ to get
>> > started
>> > in
>> > generating alerts from server?
>> >
>> > Please advise me...
>> >
>> > On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson >> >> > wrote:
>> >>
>> >> It looks like there is no handler configured with the name of
>> >> "default"
>> >> http://sensuapp.org/docs/0.16/handlers
>> >>
>> >> Good handlers for metrics are things like graphite, etc?
>> >>
>> >> On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> wrote:
>> >> > Hi,
>> >> > We are facing a problem of not getting any check
>> >> > "metrics"
>> >> > response from the client while following the tutorial at link
>> >> > Adding
>> >> > a
>> >> > Metric.
>> >> >
>> >> > Following are the outputs :-
>> >> >
>> >> > Server > tail -f /var/log/sensu/sensu-server.log
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>> >> > handler","handler_name":"default"}
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>> >> > stale clients"}
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>> >> > handler","handler_name":"default"}
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>> >> > handler","handler_name":"default"}
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>> >> > handler","handler_name":"default"}
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>> >> > check
>> >> >
>> >> >
>> >> >
>> >> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>> >> >
>> >> >
>> >> > Client > tail -f /var/log/sensu/sensu-client.log
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>> >> > check
>> >> >
>> >> >
>> >> >
>> >> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>> >> > 0.13 1425079537\nDev.load_avg.five 0.12
>> >> > 1425079537\nDev.load_avg.fifteen
>> >> > 0.22 1425079537\n\n","status":0}}}
>> >> >
>> >> >
>> >> > What are the configs that we are missing ?
>> >> >
>> >> > Thanks:-
>> >> >
>
>


#7

It is a little confusing to have both an "email" handler defined that
just uses the mail command and a "mailer.json" for using the community
mailer handler.

I recommend just using one, but I understand you are troubleshooting
at this point.

Please confirm, these configuration files are on the sensu-server,
they are valid json and readable by the sensu user, and the sensu
server has been restarted to load them?

The "unknown handler","handler_name":"email" message indicates that a
handler named email is not available, hinting that the
handler_email.json file is hasn't been loaded.

Additionally it looks like might have a typo in the email address invocation :
"command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
It looks like there are two @ signs and two double quotes? Might just
be email formatting, but that looks incorrect.

You can always verify by inspecting the sensu-server logs to confirm
your file is being loaded as intended without errors.

···

On Sun, May 3, 2015 at 12:12 PM, Srinivas Palle <srini5884@gmail.com> wrote:

I do have event_file.json file in conf.d

  "handlers": {

    "file": {

      "type": "pipe",

      "command": "/etc/sensu/plugins/event-file.rb",

      "timeout": 10,

    }

  }

}

On Mon, May 4, 2015 at 12:40 AM, Srinivas Palle <srini5884@gmail.com> wrote:

Thanks Kyle,

Unfortunately am still not getting any alerts to my mail box, am using
mailer.rb in /etc/sensu/handlers and in conf.d am using handler_email.json
and mailer.json. Will that be fine if I use both the .json fileS? Please
find configuration of handler and advise me..

I am unable to resolve this using the above docs provided.

Logs:

timestamp":"2015-05-03T19:04:35.392196+0000","level":"error","message":"unknown
handler","handler_name":"email"}

handler_email.json:

{

  "handlers": {

    "email": {

      "type": "pipe",

      "command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""

    }

  }

}

Mailer.json:

{

  "mailer": {

    "admin_gui": "http://localhost:80/",

    "mail_from": "srinivas.palle@olacabs.com",

    "mail_to": "tnt-devops@olacabs.com",

    "smtp_address": "smtp.gmail.com",

    "smtp_port": "25",

    "smtp_domain": "gmail.com"

  }

}

On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson <kyle@xkyle.com> wrote:

You could use a graphite extension to store event history.
(I'm still waiting for some sort of cool sensudb thing to be invented
for historical analysis?)

Most of the time when I need to debug a hanlder I use the sensu-server
(sometimes in combination with the sensu-client) logs to try to figure
out why a handler or event didn't do what I thought it should do.

I guess in that sense the history is stored, just in log form (not
redis and not api)

On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle <srini5884@gmail.com> >>> wrote:
> Thanks Kyle,
>
> In this case I may need to use graphite to store event history for
> certain
> period of time for debugging whenever required.
>
> Correct me if am wrong
>
> On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson <kyle@xkyle.com> wrote:
>>
>> In the API and in the dashboard (uchiwa) the last 20 status codes are
>> available in the "history" section of an event
>> http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg
>> http://sensuapp.org/docs/latest/events
>>
>> Note that because sensu is not a "green light dashboard", this event
>> history is only user-visible when the check is failing, not on checks
>> that are green.
>> In that sense they are not really "stored" (in redis) unless the check
>> is failing. Otherwise they are reported by the client to the handler
>> as they come. (as event data)
>>
>> On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle <srini5884@gmail.com> >>> >> wrote:
>> > Thanks Kyle,
>> >
>> > May I know where can we get history of events in sensu monitoring
>> > set
>> > up?
>> > Can we get history with in server?
>> >
>> > Just trying to where do all the results gets stored before handling
>> > to
>> > handler.
>> >
>> > On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson <kyle@xkyle.com> >>> >> > wrote:
>> >>
>> >> Yes, you do need to install/deploy the handler script (mailer.rb,
>> >> etc.
>> >> Usually in /etc/sensu/handlers/).
>> >> Additionally, you must add configuration for that handler
>> >> (/etc/sensu/conf.d/..) to configure that handler to perform actions
>> >> on
>> >> events (to get alerts)
>> >> Read more about how to install and configure handlers at the docs
>> >> page:
>> >>
>> >> http://sensuapp.org/docs/0.17/getting-started-with-handlers
>> >> http://sensuapp.org/docs/0.17/handlers
>> >>
>> >> On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle >>> >> >> <srini5884@gmail.com> >>> >> >> wrote:
>> >> > Hi,
>> >> >
>> >> > I do have same issue where log states unknown handler email, may
>> >> > I
>> >> > know
>> >> > the
>> >> > need of having file.rb? I do not have this file in
>> >> > /etc/sensu/handlers/,
>> >> > do
>> >> > I need to install anything or add in /etc/sensu/conf.d/ to get
>> >> > started
>> >> > in
>> >> > generating alerts from server?
>> >> >
>> >> > Please advise me...
>> >> >
>> >> > On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson >>> >> >> > wrote:
>> >> >>
>> >> >> It looks like there is no handler configured with the name of
>> >> >> "default"
>> >> >> http://sensuapp.org/docs/0.16/handlers
>> >> >>
>> >> >> Good handlers for metrics are things like graphite, etc?
>> >> >>
>> >> >> On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> wrote:
>> >> >> > Hi,
>> >> >> > We are facing a problem of not getting any check
>> >> >> > "metrics"
>> >> >> > response from the client while following the tutorial at link
>> >> >> > Adding
>> >> >> > a
>> >> >> > Metric.
>> >> >> >
>> >> >> > Following are the outputs :-
>> >> >> >
>> >> >> > Server > tail -f /var/log/sensu/sensu-server.log
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>> >> >> > handler","handler_name":"default"}
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>> >> >> > stale clients"}
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>> >> >> > handler","handler_name":"default"}
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>> >> >> > handler","handler_name":"default"}
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>> >> >> > handler","handler_name":"default"}
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>> >> >> > check
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>> >> >> >
>> >> >> >
>> >> >> > Client > tail -f /var/log/sensu/sensu-client.log
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>> >> >> > check
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>> >> >> > 0.13 1425079537\nDev.load_avg.five 0.12
>> >> >> > 1425079537\nDev.load_avg.fifteen
>> >> >> > 0.22 1425079537\n\n","status":0}}}
>> >> >> >
>> >> >> >
>> >> >> > What are the configs that we are missing ?
>> >> >> >
>> >> >> > Thanks:-
>> >> >> >
>> >
>> >
>
>


#8

(please reply-all to sensu-users so others can learn and potentially respond)

Can you confirm you are restarting the sensu server when changing
check definitions and restating the server when changing handler
definitions? (long term a I suggest using a configuration management
tool to do this for you)

Your check definition says "mail" as the handler, but your handler
json says the name is "email".

The "unknown handler" message still indicates that the "email" handler
json is not being read or parsed or something. Inspect
sensu-server.log to see why that might be the case after restarting
sensu-server.

The next error you will have to tackle is
"/etc/sensu/plugins/disk-fail.rb:64: class definition in method
body\n" from the check, but I would worry about that after you have
fixed the handler situation.

···

On Sun, May 3, 2015 at 12:35 PM, Srinivas Palle <srini5884@gmail.com> wrote:

Here is my conf.d file for disk fail check:

{

  "checks": {

    "disk-fail_check": {

      "command": "/etc/sensu/plugins/disk-fail.rb",

      "interval": 60,

      "handlers": ["default", "mail"],

      "subscribers": [ "ALL" ]

    }

  }

}

On Mon, May 4, 2015 at 1:03 AM, Srinivas Palle <srini5884@gmail.com> wrote:

I still see the same error even after making below corrections, now am
trying to use only one handler.

I hope this will be helpful...

I would like to thank you for helping me to resolve this issue.

On Mon, May 4, 2015 at 12:48 AM, Kyle Anderson <kyle@xkyle.com> wrote:

It is a little confusing to have both an "email" handler defined that
just uses the mail command and a "mailer.json" for using the community
mailer handler.

I recommend just using one, but I understand you are troubleshooting
at this point.

Please confirm, these configuration files are on the sensu-server,
they are valid json and readable by the sensu user, and the sensu
server has been restarted to load them?

The "unknown handler","handler_name":"email" message indicates that a
handler named email is not available, hinting that the
handler_email.json file is hasn't been loaded.

Additionally it looks like might have a typo in the email address
invocation :
"command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
It looks like there are two @ signs and two double quotes? Might just
be email formatting, but that looks incorrect.

You can always verify by inspecting the sensu-server logs to confirm
your file is being loaded as intended without errors.

On Sun, May 3, 2015 at 12:12 PM, Srinivas Palle <srini5884@gmail.com> >>> wrote:
> I do have event_file.json file in conf.d
>
>
> "handlers": {
>
> "file": {
>
> "type": "pipe",
>
> "command": "/etc/sensu/plugins/event-file.rb",
>
> "timeout": 10,
>
> }
>
> }
>
> }
>
>
> On Mon, May 4, 2015 at 12:40 AM, Srinivas Palle <srini5884@gmail.com> >>> > wrote:
>>
>> Thanks Kyle,
>>
>> Unfortunately am still not getting any alerts to my mail box, am using
>> mailer.rb in /etc/sensu/handlers and in conf.d am using
>> handler_email.json
>> and mailer.json. Will that be fine if I use both the .json fileS?
>> Please
>> find configuration of handler and advise me..
>>
>> I am unable to resolve this using the above docs provided.
>>
>> Logs:
>>
>>
>>
>> timestamp":"2015-05-03T19:04:35.392196+0000","level":"error","message":"unknown
>> handler","handler_name":"email"}
>>
>>
>> handler_email.json:
>>
>> {
>>
>> "handlers": {
>>
>> "email": {
>>
>> "type": "pipe",
>>
>> "command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
>>
>> }
>>
>> }
>>
>> }
>>
>>
>> Mailer.json:
>>
>>
>>
>>
>> {
>>
>> "mailer": {
>>
>> "admin_gui": "http://localhost:80/",
>>
>> "mail_from": "srinivas.palle@olacabs.com",
>>
>> "mail_to": "tnt-devops@olacabs.com",
>>
>> "smtp_address": "smtp.gmail.com",
>>
>> "smtp_port": "25",
>>
>> "smtp_domain": "gmail.com"
>>
>> }
>>
>> }
>>
>>
>> On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson <kyle@xkyle.com> wrote:
>>>
>>> You could use a graphite extension to store event history.
>>> (I'm still waiting for some sort of cool sensudb thing to be invented
>>> for historical analysis?)
>>>
>>> Most of the time when I need to debug a hanlder I use the
>>> sensu-server
>>> (sometimes in combination with the sensu-client) logs to try to
>>> figure
>>> out why a handler or event didn't do what I thought it should do.
>>>
>>> I guess in that sense the history is stored, just in log form (not
>>> redis and not api)
>>>
>>> On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle <srini5884@gmail.com> >>> >>> wrote:
>>> > Thanks Kyle,
>>> >
>>> > In this case I may need to use graphite to store event history for
>>> > certain
>>> > period of time for debugging whenever required.
>>> >
>>> > Correct me if am wrong
>>> >
>>> > On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson <kyle@xkyle.com> >>> >>> > wrote:
>>> >>
>>> >> In the API and in the dashboard (uchiwa) the last 20 status codes
>>> >> are
>>> >> available in the "history" section of an event
>>> >> http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg
>>> >> http://sensuapp.org/docs/latest/events
>>> >>
>>> >> Note that because sensu is not a "green light dashboard", this
>>> >> event
>>> >> history is only user-visible when the check is failing, not on
>>> >> checks
>>> >> that are green.
>>> >> In that sense they are not really "stored" (in redis) unless the
>>> >> check
>>> >> is failing. Otherwise they are reported by the client to the
>>> >> handler
>>> >> as they come. (as event data)
>>> >>
>>> >> On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle >>> >>> >> <srini5884@gmail.com> >>> >>> >> wrote:
>>> >> > Thanks Kyle,
>>> >> >
>>> >> > May I know where can we get history of events in sensu
>>> >> > monitoring
>>> >> > set
>>> >> > up?
>>> >> > Can we get history with in server?
>>> >> >
>>> >> > Just trying to where do all the results gets stored before
>>> >> > handling
>>> >> > to
>>> >> > handler.
>>> >> >
>>> >> > On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson <kyle@xkyle.com> >>> >>> >> > wrote:
>>> >> >>
>>> >> >> Yes, you do need to install/deploy the handler script
>>> >> >> (mailer.rb,
>>> >> >> etc.
>>> >> >> Usually in /etc/sensu/handlers/).
>>> >> >> Additionally, you must add configuration for that handler
>>> >> >> (/etc/sensu/conf.d/..) to configure that handler to perform
>>> >> >> actions
>>> >> >> on
>>> >> >> events (to get alerts)
>>> >> >> Read more about how to install and configure handlers at the
>>> >> >> docs
>>> >> >> page:
>>> >> >>
>>> >> >> http://sensuapp.org/docs/0.17/getting-started-with-handlers
>>> >> >> http://sensuapp.org/docs/0.17/handlers
>>> >> >>
>>> >> >> On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle >>> >>> >> >> <srini5884@gmail.com> >>> >>> >> >> wrote:
>>> >> >> > Hi,
>>> >> >> >
>>> >> >> > I do have same issue where log states unknown handler email,
>>> >> >> > may
>>> >> >> > I
>>> >> >> > know
>>> >> >> > the
>>> >> >> > need of having file.rb? I do not have this file in
>>> >> >> > /etc/sensu/handlers/,
>>> >> >> > do
>>> >> >> > I need to install anything or add in /etc/sensu/conf.d/ to
>>> >> >> > get
>>> >> >> > started
>>> >> >> > in
>>> >> >> > generating alerts from server?
>>> >> >> >
>>> >> >> > Please advise me...
>>> >> >> >
>>> >> >> > On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle >>> >>> >> >> > Anderson >>> >>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> It looks like there is no handler configured with the name
>>> >> >> >> of
>>> >> >> >> "default"
>>> >> >> >> http://sensuapp.org/docs/0.16/handlers
>>> >> >> >>
>>> >> >> >> Good handlers for metrics are things like graphite, etc?
>>> >> >> >>
>>> >> >> >> On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> >>> >>> >> >> >> wrote:
>>> >> >> >> > Hi,
>>> >> >> >> > We are facing a problem of not getting any
>>> >> >> >> > check
>>> >> >> >> > "metrics"
>>> >> >> >> > response from the client while following the tutorial at
>>> >> >> >> > link
>>> >> >> >> > Adding
>>> >> >> >> > a
>>> >> >> >> > Metric.
>>> >> >> >> >
>>> >> >> >> > Following are the outputs :-
>>> >> >> >> >
>>> >> >> >> > Server > tail -f /var/log/sensu/sensu-server.log
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>>> >> >> >> > handler","handler_name":"default"}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>>> >> >> >> > stale clients"}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>>> >> >> >> > handler","handler_name":"default"}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>>> >> >> >> > handler","handler_name":"default"}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>>> >> >> >> > handler","handler_name":"default"}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>>> >> >> >> > check
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > Client > tail -f /var/log/sensu/sensu-client.log
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>>> >> >> >> > check
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>>> >> >> >> > 0.13 1425079537\nDev.load_avg.five 0.12
>>> >> >> >> > 1425079537\nDev.load_avg.fifteen
>>> >> >> >> > 0.22 1425079537\n\n","status":0}}}
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > What are the configs that we are missing ?
>>> >> >> >> >
>>> >> >> >> > Thanks:-
>>> >> >> >> >
>>> >> >
>>> >> >
>>> >
>>> >
>>
>>
>


#9

(please reply-all to sensu-users so others can learn and potentially respond)

Yes, that looks good now. Your email handlers and check definitions
line up. That should have resulted in an actual email. (if mailx
worked)

Longer term you should work out why disk-fail.rb is failing on line
64. Also eventually you will want to use the mailer.rb community
handler over mailx to give you more flexibility.

Also configuration management (chef/puppet,etc) long term will allow
you to mange sensu configuration as it gets more complex without
hand-making json files, fewer typos, auto restarts, etc. I highly
recommend using a tool like this.

···

On Sun, May 3, 2015 at 1:07 PM, Srinivas Palle <srini5884@gmail.com> wrote:

Thanks Kyle,

Finally I can see no error showing. Please confirm if below log shows
everything fine or not?

{"timestamp":"2015-05-03T20:05:35.392668+0000","level":"info","message":"processing
event","event":{"id":"03f7ea89-8ca4-4ff8-9cc5-e8684ba8d245","client":{"name":"server","address":"localhost","subscriptions":["ALL"],"version":"0.17.2","timestamp":1430683520},"check":{"command":"/etc/sensu/plugins/disk-fail.rb","interval":60,"handlers":["default","email"],"subscribers":["ALL"],"name":"disk-fail_check","issued":1430683535,"executed":1430683535,"duration":0.013,"output":"/etc/sensu/plugins/disk-fail.rb:64:
class definition in method
body\n","status":1,"history":["1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1"],"total_state_change":0},"occurrences":1756,"action":"create"}}

{"timestamp":"2015-05-03T20:05:35.394632+0000","level":"info","message":"handler
output","handler":{"type":"pipe","command":"true","name":"default"},"output":}

{"timestamp":"2015-05-03T20:05:35.411406+0000","level":"info","message":"handler
output","handler":{"type":"pipe","command":"mailx -s 'sensu alert'
tnt-devops@olacabs.com","name":"email"},"output":}

On Mon, May 4, 2015 at 1:17 AM, Kyle Anderson <kyle@xkyle.com> wrote:

(please reply-all to sensu-users so others can learn and potentially
respond)

Can you confirm you are restarting the sensu server when changing
check definitions and restating the server when changing handler
definitions? (long term a I suggest using a configuration management
tool to do this for you)

Your check definition says "mail" as the handler, but your handler
json says the name is "email".

The "unknown handler" message still indicates that the "email" handler
json is not being read or parsed or something. Inspect
sensu-server.log to see why that might be the case after restarting
sensu-server.

The next error you will have to tackle is
"/etc/sensu/plugins/disk-fail.rb:64: class definition in method
body\n" from the check, but I would worry about that after you have
fixed the handler situation.

On Sun, May 3, 2015 at 12:35 PM, Srinivas Palle <srini5884@gmail.com> >> wrote:
> Here is my conf.d file for disk fail check:
>
> {
>
> "checks": {
>
> "disk-fail_check": {
>
> "command": "/etc/sensu/plugins/disk-fail.rb",
>
> "interval": 60,
>
> "handlers": ["default", "mail"],
>
> "subscribers": [ "ALL" ]
>
> }
>
> }
>
> }
>
>
> On Mon, May 4, 2015 at 1:03 AM, Srinivas Palle <srini5884@gmail.com> >> > wrote:
>>
>> I still see the same error even after making below corrections, now am
>> trying to use only one handler.
>>
>> I hope this will be helpful...
>>
>> I would like to thank you for helping me to resolve this issue.
>>
>> On Mon, May 4, 2015 at 12:48 AM, Kyle Anderson <kyle@xkyle.com> wrote:
>>>
>>> It is a little confusing to have both an "email" handler defined that
>>> just uses the mail command and a "mailer.json" for using the community
>>> mailer handler.
>>>
>>> I recommend just using one, but I understand you are troubleshooting
>>> at this point.
>>>
>>> Please confirm, these configuration files are on the sensu-server,
>>> they are valid json and readable by the sensu user, and the sensu
>>> server has been restarted to load them?
>>>
>>> The "unknown handler","handler_name":"email" message indicates that a
>>> handler named email is not available, hinting that the
>>> handler_email.json file is hasn't been loaded.
>>>
>>> Additionally it looks like might have a typo in the email address
>>> invocation :
>>> "command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
>>> It looks like there are two @ signs and two double quotes? Might just
>>> be email formatting, but that looks incorrect.
>>>
>>> You can always verify by inspecting the sensu-server logs to confirm
>>> your file is being loaded as intended without errors.
>>>
>>> On Sun, May 3, 2015 at 12:12 PM, Srinivas Palle <srini5884@gmail.com> >> >>> wrote:
>>> > I do have event_file.json file in conf.d
>>> >
>>> >
>>> > "handlers": {
>>> >
>>> > "file": {
>>> >
>>> > "type": "pipe",
>>> >
>>> > "command": "/etc/sensu/plugins/event-file.rb",
>>> >
>>> > "timeout": 10,
>>> >
>>> > }
>>> >
>>> > }
>>> >
>>> > }
>>> >
>>> >
>>> > On Mon, May 4, 2015 at 12:40 AM, Srinivas Palle >> >>> > <srini5884@gmail.com> >> >>> > wrote:
>>> >>
>>> >> Thanks Kyle,
>>> >>
>>> >> Unfortunately am still not getting any alerts to my mail box, am
>>> >> using
>>> >> mailer.rb in /etc/sensu/handlers and in conf.d am using
>>> >> handler_email.json
>>> >> and mailer.json. Will that be fine if I use both the .json fileS?
>>> >> Please
>>> >> find configuration of handler and advise me..
>>> >>
>>> >> I am unable to resolve this using the above docs provided.
>>> >>
>>> >> Logs:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> timestamp":"2015-05-03T19:04:35.392196+0000","level":"error","message":"unknown
>>> >> handler","handler_name":"email"}
>>> >>
>>> >>
>>> >> handler_email.json:
>>> >>
>>> >> {
>>> >>
>>> >> "handlers": {
>>> >>
>>> >> "email": {
>>> >>
>>> >> "type": "pipe",
>>> >>
>>> >> "command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
>>> >>
>>> >> }
>>> >>
>>> >> }
>>> >>
>>> >> }
>>> >>
>>> >>
>>> >> Mailer.json:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> {
>>> >>
>>> >> "mailer": {
>>> >>
>>> >> "admin_gui": "http://localhost:80/",
>>> >>
>>> >> "mail_from": "srinivas.palle@olacabs.com",
>>> >>
>>> >> "mail_to": "tnt-devops@olacabs.com",
>>> >>
>>> >> "smtp_address": "smtp.gmail.com",
>>> >>
>>> >> "smtp_port": "25",
>>> >>
>>> >> "smtp_domain": "gmail.com"
>>> >>
>>> >> }
>>> >>
>>> >> }
>>> >>
>>> >>
>>> >> On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson <kyle@xkyle.com> >> >>> >> wrote:
>>> >>>
>>> >>> You could use a graphite extension to store event history.
>>> >>> (I'm still waiting for some sort of cool sensudb thing to be
>>> >>> invented
>>> >>> for historical analysis?)
>>> >>>
>>> >>> Most of the time when I need to debug a hanlder I use the
>>> >>> sensu-server
>>> >>> (sometimes in combination with the sensu-client) logs to try to
>>> >>> figure
>>> >>> out why a handler or event didn't do what I thought it should do.
>>> >>>
>>> >>> I guess in that sense the history is stored, just in log form (not
>>> >>> redis and not api)
>>> >>>
>>> >>> On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle >> >>> >>> <srini5884@gmail.com> >> >>> >>> wrote:
>>> >>> > Thanks Kyle,
>>> >>> >
>>> >>> > In this case I may need to use graphite to store event history
>>> >>> > for
>>> >>> > certain
>>> >>> > period of time for debugging whenever required.
>>> >>> >
>>> >>> > Correct me if am wrong
>>> >>> >
>>> >>> > On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson <kyle@xkyle.com> >> >>> >>> > wrote:
>>> >>> >>
>>> >>> >> In the API and in the dashboard (uchiwa) the last 20 status
>>> >>> >> codes
>>> >>> >> are
>>> >>> >> available in the "history" section of an event
>>> >>> >> http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg
>>> >>> >> http://sensuapp.org/docs/latest/events
>>> >>> >>
>>> >>> >> Note that because sensu is not a "green light dashboard", this
>>> >>> >> event
>>> >>> >> history is only user-visible when the check is failing, not on
>>> >>> >> checks
>>> >>> >> that are green.
>>> >>> >> In that sense they are not really "stored" (in redis) unless
>>> >>> >> the
>>> >>> >> check
>>> >>> >> is failing. Otherwise they are reported by the client to the
>>> >>> >> handler
>>> >>> >> as they come. (as event data)
>>> >>> >>
>>> >>> >> On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle >> >>> >>> >> <srini5884@gmail.com> >> >>> >>> >> wrote:
>>> >>> >> > Thanks Kyle,
>>> >>> >> >
>>> >>> >> > May I know where can we get history of events in sensu
>>> >>> >> > monitoring
>>> >>> >> > set
>>> >>> >> > up?
>>> >>> >> > Can we get history with in server?
>>> >>> >> >
>>> >>> >> > Just trying to where do all the results gets stored before
>>> >>> >> > handling
>>> >>> >> > to
>>> >>> >> > handler.
>>> >>> >> >
>>> >>> >> > On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson >> >>> >>> >> > <kyle@xkyle.com> >> >>> >>> >> > wrote:
>>> >>> >> >>
>>> >>> >> >> Yes, you do need to install/deploy the handler script
>>> >>> >> >> (mailer.rb,
>>> >>> >> >> etc.
>>> >>> >> >> Usually in /etc/sensu/handlers/).
>>> >>> >> >> Additionally, you must add configuration for that handler
>>> >>> >> >> (/etc/sensu/conf.d/..) to configure that handler to perform
>>> >>> >> >> actions
>>> >>> >> >> on
>>> >>> >> >> events (to get alerts)
>>> >>> >> >> Read more about how to install and configure handlers at the
>>> >>> >> >> docs
>>> >>> >> >> page:
>>> >>> >> >>
>>> >>> >> >> http://sensuapp.org/docs/0.17/getting-started-with-handlers
>>> >>> >> >> http://sensuapp.org/docs/0.17/handlers
>>> >>> >> >>
>>> >>> >> >> On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle >> >>> >>> >> >> <srini5884@gmail.com> >> >>> >>> >> >> wrote:
>>> >>> >> >> > Hi,
>>> >>> >> >> >
>>> >>> >> >> > I do have same issue where log states unknown handler
>>> >>> >> >> > email,
>>> >>> >> >> > may
>>> >>> >> >> > I
>>> >>> >> >> > know
>>> >>> >> >> > the
>>> >>> >> >> > need of having file.rb? I do not have this file in
>>> >>> >> >> > /etc/sensu/handlers/,
>>> >>> >> >> > do
>>> >>> >> >> > I need to install anything or add in /etc/sensu/conf.d/ to
>>> >>> >> >> > get
>>> >>> >> >> > started
>>> >>> >> >> > in
>>> >>> >> >> > generating alerts from server?
>>> >>> >> >> >
>>> >>> >> >> > Please advise me...
>>> >>> >> >> >
>>> >>> >> >> > On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle >> >>> >>> >> >> > Anderson >> >>> >>> >> >> > wrote:
>>> >>> >> >> >>
>>> >>> >> >> >> It looks like there is no handler configured with the
>>> >>> >> >> >> name
>>> >>> >> >> >> of
>>> >>> >> >> >> "default"
>>> >>> >> >> >> http://sensuapp.org/docs/0.16/handlers
>>> >>> >> >> >>
>>> >>> >> >> >> Good handlers for metrics are things like graphite, etc?
>>> >>> >> >> >>
>>> >>> >> >> >> On Fri, Feb 27, 2015 at 3:28 PM, <max2su...@gmail.com> >> >>> >>> >> >> >> wrote:
>>> >>> >> >> >> > Hi,
>>> >>> >> >> >> > We are facing a problem of not getting any
>>> >>> >> >> >> > check
>>> >>> >> >> >> > "metrics"
>>> >>> >> >> >> > response from the client while following the tutorial
>>> >>> >> >> >> > at
>>> >>> >> >> >> > link
>>> >>> >> >> >> > Adding
>>> >>> >> >> >> > a
>>> >>> >> >> >> > Metric.
>>> >>> >> >> >> >
>>> >>> >> >> >> > Following are the outputs :-
>>> >>> >> >> >> >
>>> >>> >> >> >> > Server > tail -f /var/log/sensu/sensu-server.log
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>>> >>> >> >> >> > handler","handler_name":"default"}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>>> >>> >> >> >> > stale clients"}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>>> >>> >> >> >> > handler","handler_name":"default"}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>>> >>> >> >> >> > handler","handler_name":"default"}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>>> >>> >> >> >> > handler","handler_name":"default"}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>>> >>> >> >> >> > check
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > Client > tail -f /var/log/sensu/sensu-client.log
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>>> >>> >> >> >> > check
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>>> >>> >> >> >> > 0.13 1425079537\nDev.load_avg.five 0.12
>>> >>> >> >> >> > 1425079537\nDev.load_avg.fifteen
>>> >>> >> >> >> > 0.22 1425079537\n\n","status":0}}}
>>> >>> >> >> >> >
>>> >>> >> >> >> >
>>> >>> >> >> >> > What are the configs that we are missing ?
>>> >>> >> >> >> >
>>> >>> >> >> >> > Thanks:-
>>> >>> >> >> >> >
>>> >>> >> >
>>> >>> >> >
>>> >>> >
>>> >>> >
>>> >>
>>> >>
>>> >
>>
>>
>


#10

(please reply-all to sensu-users so others can learn and potentially respond)

Possibly.
You should first verify either of those commands work before
configuring sensu to use it.

echo test | mail -s "mail command test" tnt-devops@olacabs.com
echo test | mailx -s "mailx command test" tnt-devops@olacabs.com

etc.

Both of those depend on having a local mail server running and
configured. (I think?)
I think the ruby based mail handler sends out via stmp directly?

···

On Sun, May 3, 2015 at 2:09 PM, Srinivas Palle <srini5884@gmail.com> wrote:

Thanks Kyle,

As of now I didn't get any alert to my mailbox thought I configured for
minimum thresolds, looks like mailx command not working here.

Shall I use mail -s command instead?

Also, I would work on automation on priority to make my life easier.

On Mon, May 4, 2015 at 1:55 AM, Kyle Anderson <kyle@xkyle.com> wrote:

(please reply-all to sensu-users so others can learn and potentially
respond)

Yes, that looks good now. Your email handlers and check definitions
line up. That should have resulted in an actual email. (if mailx
worked)

Longer term you should work out why disk-fail.rb is failing on line
64. Also eventually you will want to use the mailer.rb community
handler over mailx to give you more flexibility.

Also configuration management (chef/puppet,etc) long term will allow
you to mange sensu configuration as it gets more complex without
hand-making json files, fewer typos, auto restarts, etc. I highly
recommend using a tool like this.

On Sun, May 3, 2015 at 1:07 PM, Srinivas Palle <srini5884@gmail.com> >> wrote:
> Thanks Kyle,
>
> Finally I can see no error showing. Please confirm if below log shows
> everything fine or not?
>
>
> {"timestamp":"2015-05-03T20:05:35.392668+0000","level":"info","message":"processing
>
> event","event":{"id":"03f7ea89-8ca4-4ff8-9cc5-e8684ba8d245","client":{"name":"server","address":"localhost","subscriptions":["ALL"],"version":"0.17.2","timestamp":1430683520},"check":{"command":"/etc/sensu/plugins/disk-fail.rb","interval":60,"handlers":["default","email"],"subscribers":["ALL"],"name":"disk-fail_check","issued":1430683535,"executed":1430683535,"duration":0.013,"output":"/etc/sensu/plugins/disk-fail.rb:64:
> class definition in method
>
> body\n","status":1,"history":["1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1","1"],"total_state_change":0},"occurrences":1756,"action":"create"}}
>
>
> {"timestamp":"2015-05-03T20:05:35.394632+0000","level":"info","message":"handler
>
> output","handler":{"type":"pipe","command":"true","name":"default"},"output":}
>
>
> {"timestamp":"2015-05-03T20:05:35.411406+0000","level":"info","message":"handler
> output","handler":{"type":"pipe","command":"mailx -s 'sensu alert'
> tnt-devops@olacabs.com","name":"email"},"output":}
>
>
> On Mon, May 4, 2015 at 1:17 AM, Kyle Anderson <kyle@xkyle.com> wrote:
>>
>> (please reply-all to sensu-users so others can learn and potentially
>> respond)
>>
>> Can you confirm you are restarting the sensu server when changing
>> check definitions and restating the server when changing handler
>> definitions? (long term a I suggest using a configuration management
>> tool to do this for you)
>>
>> Your check definition says "mail" as the handler, but your handler
>> json says the name is "email".
>>
>> The "unknown handler" message still indicates that the "email" handler
>> json is not being read or parsed or something. Inspect
>> sensu-server.log to see why that might be the case after restarting
>> sensu-server.
>>
>>
>> The next error you will have to tackle is
>> "/etc/sensu/plugins/disk-fail.rb:64: class definition in method
>> body\n" from the check, but I would worry about that after you have
>> fixed the handler situation.
>>
>> On Sun, May 3, 2015 at 12:35 PM, Srinivas Palle <srini5884@gmail.com> >> >> wrote:
>> > Here is my conf.d file for disk fail check:
>> >
>> > {
>> >
>> > "checks": {
>> >
>> > "disk-fail_check": {
>> >
>> > "command": "/etc/sensu/plugins/disk-fail.rb",
>> >
>> > "interval": 60,
>> >
>> > "handlers": ["default", "mail"],
>> >
>> > "subscribers": [ "ALL" ]
>> >
>> > }
>> >
>> > }
>> >
>> > }
>> >
>> >
>> > On Mon, May 4, 2015 at 1:03 AM, Srinivas Palle <srini5884@gmail.com> >> >> > wrote:
>> >>
>> >> I still see the same error even after making below corrections, now
>> >> am
>> >> trying to use only one handler.
>> >>
>> >> I hope this will be helpful...
>> >>
>> >> I would like to thank you for helping me to resolve this issue.
>> >>
>> >> On Mon, May 4, 2015 at 12:48 AM, Kyle Anderson <kyle@xkyle.com> >> >> >> wrote:
>> >>>
>> >>> It is a little confusing to have both an "email" handler defined
>> >>> that
>> >>> just uses the mail command and a "mailer.json" for using the
>> >>> community
>> >>> mailer handler.
>> >>>
>> >>> I recommend just using one, but I understand you are
>> >>> troubleshooting
>> >>> at this point.
>> >>>
>> >>> Please confirm, these configuration files are on the sensu-server,
>> >>> they are valid json and readable by the sensu user, and the sensu
>> >>> server has been restarted to load them?
>> >>>
>> >>> The "unknown handler","handler_name":"email" message indicates that
>> >>> a
>> >>> handler named email is not available, hinting that the
>> >>> handler_email.json file is hasn't been loaded.
>> >>>
>> >>> Additionally it looks like might have a typo in the email address
>> >>> invocation :
>> >>> "command": "mailx -s 'sensu alert' tnt-devops@@olacabs.com""
>> >>> It looks like there are two @ signs and two double quotes? Might
>> >>> just
>> >>> be email formatting, but that looks incorrect.
>> >>>
>> >>> You can always verify by inspecting the sensu-server logs to
>> >>> confirm
>> >>> your file is being loaded as intended without errors.
>> >>>
>> >>> On Sun, May 3, 2015 at 12:12 PM, Srinivas Palle >> >> >>> <srini5884@gmail.com> >> >> >>> wrote:
>> >>> > I do have event_file.json file in conf.d
>> >>> >
>> >>> >
>> >>> > "handlers": {
>> >>> >
>> >>> > "file": {
>> >>> >
>> >>> > "type": "pipe",
>> >>> >
>> >>> > "command": "/etc/sensu/plugins/event-file.rb",
>> >>> >
>> >>> > "timeout": 10,
>> >>> >
>> >>> > }
>> >>> >
>> >>> > }
>> >>> >
>> >>> > }
>> >>> >
>> >>> >
>> >>> > On Mon, May 4, 2015 at 12:40 AM, Srinivas Palle >> >> >>> > <srini5884@gmail.com> >> >> >>> > wrote:
>> >>> >>
>> >>> >> Thanks Kyle,
>> >>> >>
>> >>> >> Unfortunately am still not getting any alerts to my mail box, am
>> >>> >> using
>> >>> >> mailer.rb in /etc/sensu/handlers and in conf.d am using
>> >>> >> handler_email.json
>> >>> >> and mailer.json. Will that be fine if I use both the .json
>> >>> >> fileS?
>> >>> >> Please
>> >>> >> find configuration of handler and advise me..
>> >>> >>
>> >>> >> I am unable to resolve this using the above docs provided.
>> >>> >>
>> >>> >> Logs:
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> timestamp":"2015-05-03T19:04:35.392196+0000","level":"error","message":"unknown
>> >>> >> handler","handler_name":"email"}
>> >>> >>
>> >>> >>
>> >>> >> handler_email.json:
>> >>> >>
>> >>> >> {
>> >>> >>
>> >>> >> "handlers": {
>> >>> >>
>> >>> >> "email": {
>> >>> >>
>> >>> >> "type": "pipe",
>> >>> >>
>> >>> >> "command": "mailx -s 'sensu alert'
>> >>> >> tnt-devops@@olacabs.com""
>> >>> >>
>> >>> >> }
>> >>> >>
>> >>> >> }
>> >>> >>
>> >>> >> }
>> >>> >>
>> >>> >>
>> >>> >> Mailer.json:
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> {
>> >>> >>
>> >>> >> "mailer": {
>> >>> >>
>> >>> >> "admin_gui": "http://localhost:80/",
>> >>> >>
>> >>> >> "mail_from": "srinivas.palle@olacabs.com",
>> >>> >>
>> >>> >> "mail_to": "tnt-devops@olacabs.com",
>> >>> >>
>> >>> >> "smtp_address": "smtp.gmail.com",
>> >>> >>
>> >>> >> "smtp_port": "25",
>> >>> >>
>> >>> >> "smtp_domain": "gmail.com"
>> >>> >>
>> >>> >> }
>> >>> >>
>> >>> >> }
>> >>> >>
>> >>> >>
>> >>> >> On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson <kyle@xkyle.com> >> >> >>> >> wrote:
>> >>> >>>
>> >>> >>> You could use a graphite extension to store event history.
>> >>> >>> (I'm still waiting for some sort of cool sensudb thing to be
>> >>> >>> invented
>> >>> >>> for historical analysis?)
>> >>> >>>
>> >>> >>> Most of the time when I need to debug a hanlder I use the
>> >>> >>> sensu-server
>> >>> >>> (sometimes in combination with the sensu-client) logs to try to
>> >>> >>> figure
>> >>> >>> out why a handler or event didn't do what I thought it should
>> >>> >>> do.
>> >>> >>>
>> >>> >>> I guess in that sense the history is stored, just in log form
>> >>> >>> (not
>> >>> >>> redis and not api)
>> >>> >>>
>> >>> >>> On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle >> >> >>> >>> <srini5884@gmail.com> >> >> >>> >>> wrote:
>> >>> >>> > Thanks Kyle,
>> >>> >>> >
>> >>> >>> > In this case I may need to use graphite to store event
>> >>> >>> > history
>> >>> >>> > for
>> >>> >>> > certain
>> >>> >>> > period of time for debugging whenever required.
>> >>> >>> >
>> >>> >>> > Correct me if am wrong
>> >>> >>> >
>> >>> >>> > On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson >> >> >>> >>> > <kyle@xkyle.com> >> >> >>> >>> > wrote:
>> >>> >>> >>
>> >>> >>> >> In the API and in the dashboard (uchiwa) the last 20 status
>> >>> >>> >> codes
>> >>> >>> >> are
>> >>> >>> >> available in the "history" section of an event
>> >>> >>> >>
>> >>> >>> >> http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg
>> >>> >>> >> http://sensuapp.org/docs/latest/events
>> >>> >>> >>
>> >>> >>> >> Note that because sensu is not a "green light dashboard",
>> >>> >>> >> this
>> >>> >>> >> event
>> >>> >>> >> history is only user-visible when the check is failing, not
>> >>> >>> >> on
>> >>> >>> >> checks
>> >>> >>> >> that are green.
>> >>> >>> >> In that sense they are not really "stored" (in redis) unless
>> >>> >>> >> the
>> >>> >>> >> check
>> >>> >>> >> is failing. Otherwise they are reported by the client to the
>> >>> >>> >> handler
>> >>> >>> >> as they come. (as event data)
>> >>> >>> >>
>> >>> >>> >> On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle >> >> >>> >>> >> <srini5884@gmail.com> >> >> >>> >>> >> wrote:
>> >>> >>> >> > Thanks Kyle,
>> >>> >>> >> >
>> >>> >>> >> > May I know where can we get history of events in sensu
>> >>> >>> >> > monitoring
>> >>> >>> >> > set
>> >>> >>> >> > up?
>> >>> >>> >> > Can we get history with in server?
>> >>> >>> >> >
>> >>> >>> >> > Just trying to where do all the results gets stored before
>> >>> >>> >> > handling
>> >>> >>> >> > to
>> >>> >>> >> > handler.
>> >>> >>> >> >
>> >>> >>> >> > On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson >> >> >>> >>> >> > <kyle@xkyle.com> >> >> >>> >>> >> > wrote:
>> >>> >>> >> >>
>> >>> >>> >> >> Yes, you do need to install/deploy the handler script
>> >>> >>> >> >> (mailer.rb,
>> >>> >>> >> >> etc.
>> >>> >>> >> >> Usually in /etc/sensu/handlers/).
>> >>> >>> >> >> Additionally, you must add configuration for that handler
>> >>> >>> >> >> (/etc/sensu/conf.d/..) to configure that handler to
>> >>> >>> >> >> perform
>> >>> >>> >> >> actions
>> >>> >>> >> >> on
>> >>> >>> >> >> events (to get alerts)
>> >>> >>> >> >> Read more about how to install and configure handlers at
>> >>> >>> >> >> the
>> >>> >>> >> >> docs
>> >>> >>> >> >> page:
>> >>> >>> >> >>
>> >>> >>> >> >>
>> >>> >>> >> >> http://sensuapp.org/docs/0.17/getting-started-with-handlers
>> >>> >>> >> >> http://sensuapp.org/docs/0.17/handlers
>> >>> >>> >> >>
>> >>> >>> >> >> On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle >> >> >>> >>> >> >> <srini5884@gmail.com> >> >> >>> >>> >> >> wrote:
>> >>> >>> >> >> > Hi,
>> >>> >>> >> >> >
>> >>> >>> >> >> > I do have same issue where log states unknown handler
>> >>> >>> >> >> > email,
>> >>> >>> >> >> > may
>> >>> >>> >> >> > I
>> >>> >>> >> >> > know
>> >>> >>> >> >> > the
>> >>> >>> >> >> > need of having file.rb? I do not have this file in
>> >>> >>> >> >> > /etc/sensu/handlers/,
>> >>> >>> >> >> > do
>> >>> >>> >> >> > I need to install anything or add in /etc/sensu/conf.d/
>> >>> >>> >> >> > to
>> >>> >>> >> >> > get
>> >>> >>> >> >> > started
>> >>> >>> >> >> > in
>> >>> >>> >> >> > generating alerts from server?
>> >>> >>> >> >> >
>> >>> >>> >> >> > Please advise me...
>> >>> >>> >> >> >
>> >>> >>> >> >> > On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle >> >> >>> >>> >> >> > Anderson >> >> >>> >>> >> >> > wrote:
>> >>> >>> >> >> >>
>> >>> >>> >> >> >> It looks like there is no handler configured with the
>> >>> >>> >> >> >> name
>> >>> >>> >> >> >> of
>> >>> >>> >> >> >> "default"
>> >>> >>> >> >> >> http://sensuapp.org/docs/0.16/handlers
>> >>> >>> >> >> >>
>> >>> >>> >> >> >> Good handlers for metrics are things like graphite,
>> >>> >>> >> >> >> etc?
>> >>> >>> >> >> >>
>> >>> >>> >> >> >> On Fri, Feb 27, 2015 at 3:28 PM, >> >> >>> >>> >> >> >> <max2su...@gmail.com> >> >> >>> >>> >> >> >> wrote:
>> >>> >>> >> >> >> > Hi,
>> >>> >>> >> >> >> > We are facing a problem of not getting
>> >>> >>> >> >> >> > any
>> >>> >>> >> >> >> > check
>> >>> >>> >> >> >> > "metrics"
>> >>> >>> >> >> >> > response from the client while following the
>> >>> >>> >> >> >> > tutorial
>> >>> >>> >> >> >> > at
>> >>> >>> >> >> >> > link
>> >>> >>> >> >> >> > Adding
>> >>> >>> >> >> >> > a
>> >>> >>> >> >> >> > Metric.
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > Following are the outputs :-
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > Server > tail -f /var/log/sensu/sensu-server.log
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
>> >>> >>> >> >> >> > handler","handler_name":"default"}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
>> >>> >>> >> >> >> > stale clients"}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
>> >>> >>> >> >> >> > handler","handler_name":"default"}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
>> >>> >>> >> >> >> > handler","handler_name":"default"}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
>> >>> >>> >> >> >> > handler","handler_name":"default"}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
>> >>> >>> >> >> >> > check
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > Client > tail -f /var/log/sensu/sensu-client.log
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > {"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
>> >>> >>> >> >> >> > check
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
>> >>> >>> >> >> >> > 0.13 1425079537\nDev.load_avg.five 0.12
>> >>> >>> >> >> >> > 1425079537\nDev.load_avg.fifteen
>> >>> >>> >> >> >> > 0.22 1425079537\n\n","status":0}}}
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > What are the configs that we are missing ?
>> >>> >>> >> >> >> >
>> >>> >>> >> >> >> > Thanks:-
>> >>> >>> >> >> >> >
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >
>> >>
>> >>
>> >
>
>


#11

I still do not see any mails alerting me… I do have postfix configured and can use it to send test mails to local mailbox.

···

On Mon, May 4, 2015 at 3:40 AM, Kyle Anderson kyle@xkyle.com wrote:

(please reply-all to sensu-users so others can learn and potentially respond)

Possibly.

You should first verify either of those commands work before

configuring sensu to use it.

echo test | mail -s “mail command test” tnt-devops@olacabs.com

echo test | mailx -s “mailx command test” tnt-devops@olacabs.com

etc.

Both of those depend on having a local mail server running and

configured. (I think?)

I think the ruby based mail handler sends out via stmp directly?

On Sun, May 3, 2015 at 2:09 PM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

As of now I didn’t get any alert to my mailbox thought I configured for

minimum thresolds, looks like mailx command not working here.

Shall I use mail -s command instead?

Also, I would work on automation on priority to make my life easier.

On Mon, May 4, 2015 at 1:55 AM, Kyle Anderson kyle@xkyle.com wrote:

(please reply-all to sensu-users so others can learn and potentially

respond)

Yes, that looks good now. Your email handlers and check definitions

line up. That should have resulted in an actual email. (if mailx

worked)

Longer term you should work out why disk-fail.rb is failing on line

  1. Also eventually you will want to use the mailer.rb community

handler over mailx to give you more flexibility.

Also configuration management (chef/puppet,etc) long term will allow

you to mange sensu configuration as it gets more complex without

hand-making json files, fewer typos, auto restarts, etc. I highly

recommend using a tool like this.

On Sun, May 3, 2015 at 1:07 PM, Srinivas Palle srini5884@gmail.com

wrote:

Thanks Kyle,

Finally I can see no error showing. Please confirm if below log shows

everything fine or not?

{“timestamp”:“2015-05-03T20:05:35.392668+0000”,“level”:“info”,“message”:"processing

event",“event”:{“id”:“03f7ea89-8ca4-4ff8-9cc5-e8684ba8d245”,“client”:{“name”:“server”,“address”:“localhost”,“subscriptions”:[“ALL”],“version”:“0.17.2”,“timestamp”:1430683520},“check”:{“command”:"/etc/sensu/plugins/disk-fail.rb",“interval”:60,“handlers”:[“default”,“email”],“subscribers”:[“ALL”],“name”:“disk-fail_check”,“issued”:1430683535,“executed”:1430683535,“duration”:0.013,“output”:"/etc/sensu/plugins/disk-fail.rb:64:

class definition in method

body\n",“status”:1,“history”:[“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”,“1”],“total_state_change”:0},“occurrences”:1756,“action”:“create”}}

{“timestamp”:“2015-05-03T20:05:35.394632+0000”,“level”:“info”,“message”:"handler

output",“handler”:{“type”:“pipe”,“command”:“true”,“name”:“default”},“output”:}

{“timestamp”:“2015-05-03T20:05:35.411406+0000”,“level”:“info”,“message”:"handler

output",“handler”:{“type”:“pipe”,“command”:"mailx -s ‘sensu alert’

tnt-devops@olacabs.com",“name”:“email”},“output”:}

On Mon, May 4, 2015 at 1:17 AM, Kyle Anderson kyle@xkyle.com wrote:

(please reply-all to sensu-users so others can learn and potentially

respond)

Can you confirm you are restarting the sensu server when changing

check definitions and restating the server when changing handler

definitions? (long term a I suggest using a configuration management

tool to do this for you)

Your check definition says “mail” as the handler, but your handler

json says the name is “email”.

The “unknown handler” message still indicates that the “email” handler

json is not being read or parsed or something. Inspect

sensu-server.log to see why that might be the case after restarting

sensu-server.

The next error you will have to tackle is

"/etc/sensu/plugins/disk-fail.rb:64: class definition in method

body\n" from the check, but I would worry about that after you have

fixed the handler situation.

On Sun, May 3, 2015 at 12:35 PM, Srinivas Palle srini5884@gmail.com

wrote:

Here is my conf.d file for disk fail check:

{

“checks”: {

"disk-fail_check": {
  "command": "/etc/sensu/plugins/disk-fail.rb",
  "interval": 60,
  "handlers": ["default", "mail"],
  "subscribers": [ "ALL" ]
}

}

}

On Mon, May 4, 2015 at 1:03 AM, Srinivas Palle srini5884@gmail.com

wrote:

I still see the same error even after making below corrections, now

am

trying to use only one handler.

I hope this will be helpful…

I would like to thank you for helping me to resolve this issue.

On Mon, May 4, 2015 at 12:48 AM, Kyle Anderson kyle@xkyle.com

wrote:

It is a little confusing to have both an “email” handler defined

that

just uses the mail command and a “mailer.json” for using the

community

mailer handler.

I recommend just using one, but I understand you are

troubleshooting

at this point.

Please confirm, these configuration files are on the sensu-server,

they are valid json and readable by the sensu user, and the sensu

server has been restarted to load them?

The “unknown handler”,“handler_name”:“email” message indicates that

a

handler named email is not available, hinting that the

handler_email.json file is hasn’t been loaded.

Additionally it looks like might have a typo in the email address

invocation :

“command”: “mailx -s ‘sensu alert’ tnt-devops@@olacabs.com”"

It looks like there are two @ signs and two double quotes? Might

just

be email formatting, but that looks incorrect.

You can always verify by inspecting the sensu-server logs to

confirm

your file is being loaded as intended without errors.

On Sun, May 3, 2015 at 12:12 PM, Srinivas Palle

srini5884@gmail.com

wrote:

I do have event_file.json file in conf.d

“handlers”: {

"file": {
  "type": "pipe",
  "command": "/etc/sensu/plugins/event-file.rb",
  "timeout": 10,
}

}

}

On Mon, May 4, 2015 at 12:40 AM, Srinivas Palle

srini5884@gmail.com

wrote:

Thanks Kyle,

Unfortunately am still not getting any alerts to my mail box, am

using

mailer.rb in /etc/sensu/handlers and in conf.d am using

handler_email.json

and mailer.json. Will that be fine if I use both the .json

fileS?

Please

find configuration of handler and advise me…

I am unable to resolve this using the above docs provided.

Logs:

timestamp":“2015-05-03T19:04:35.392196+0000”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“email”}

handler_email.json:

{

“handlers”: {

"email": {
  "type": "pipe",
  "command": "mailx -s 'sensu alert'

tnt-devops@@olacabs.com""

}

}

}

Mailer.json:

{

“mailer”: {

"admin_gui": "[http://localhost:80/](http://localhost:80/)",
"mail_from": "srinivas.palle@olacabs.com",
"mail_to": "tnt-devops@olacabs.com",
"smtp_address": "[smtp.gmail.com](http://smtp.gmail.com)",
"smtp_port": "25",
"smtp_domain": "[gmail.com](http://gmail.com)"

}

}

On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson kyle@xkyle.com

wrote:

You could use a graphite extension to store event history.

(I’m still waiting for some sort of cool sensudb thing to be

invented

for historical analysis?)

Most of the time when I need to debug a hanlder I use the

sensu-server

(sometimes in combination with the sensu-client) logs to try to

figure

out why a handler or event didn’t do what I thought it should

do.

I guess in that sense the history is stored, just in log form

(not

redis and not api)

On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle

srini5884@gmail.com

wrote:

Thanks Kyle,

In this case I may need to use graphite to store event

history

for

certain

period of time for debugging whenever required.

Correct me if am wrong

On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson

kyle@xkyle.com

wrote:

In the API and in the dashboard (uchiwa) the last 20 status

codes

are

available in the “history” section of an event

http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg

http://sensuapp.org/docs/latest/events

Note that because sensu is not a “green light dashboard”,

this

event

history is only user-visible when the check is failing, not

on

checks

that are green.

In that sense they are not really “stored” (in redis) unless

the

check

is failing. Otherwise they are reported by the client to the

handler

as they come. (as event data)

On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle

srini5884@gmail.com

wrote:

Thanks Kyle,

May I know where can we get history of events in sensu

monitoring

set

up?

Can we get history with in server?

Just trying to where do all the results gets stored before

handling

to

handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson

kyle@xkyle.com

wrote:

Yes, you do need to install/deploy the handler script

(mailer.rb,

etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to

perform

actions

on

events (to get alerts)

Read more about how to install and configure handlers at

the

docs

page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle

srini5884@gmail.com

wrote:

Hi,

I do have same issue where log states unknown handler

email,

may

I

know

the

need of having file.rb? I do not have this file in

/etc/sensu/handlers/,

do

I need to install anything or add in /etc/sensu/conf.d/

to

get

started

in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle

Anderson

wrote:

It looks like there is no handler configured with the

name

of

“default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite,

etc?

On Fri, Feb 27, 2015 at 3:28 PM,

max2su...@gmail.com

wrote:

Hi,

       We are facing a problem of not getting

any

check

“metrics”

response from the client while following the

tutorial

at

link

Adding

a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12

1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#12

Please ignore my earlier mail, it seems like issue with our internal DL’s not allowed in internet…

Thanks Kyle for your support…

···

On Mon, May 4, 2015 at 12:32 AM, Kyle Anderson kyle@xkyle.com wrote:

You could use a graphite extension to store event history.

(I’m still waiting for some sort of cool sensudb thing to be invented

for historical analysis?)

Most of the time when I need to debug a hanlder I use the sensu-server

(sometimes in combination with the sensu-client) logs to try to figure

out why a handler or event didn’t do what I thought it should do.

I guess in that sense the history is stored, just in log form (not

redis and not api)

On Sun, May 3, 2015 at 11:54 AM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

In this case I may need to use graphite to store event history for certain

period of time for debugging whenever required.

Correct me if am wrong

On Mon, May 4, 2015 at 12:12 AM, Kyle Anderson kyle@xkyle.com wrote:

In the API and in the dashboard (uchiwa) the last 20 status codes are

available in the “history” section of an event

http://static.oschina.net/uploads/img/201405/19063539_ovDI.jpg

http://sensuapp.org/docs/latest/events

Note that because sensu is not a “green light dashboard”, this event

history is only user-visible when the check is failing, not on checks

that are green.

In that sense they are not really “stored” (in redis) unless the check

is failing. Otherwise they are reported by the client to the handler

as they come. (as event data)

On Sun, May 3, 2015 at 11:12 AM, Srinivas Palle srini5884@gmail.com

wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set

up?

Can we get history with in server?

Just trying to where do all the results gets stored before handling to

handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com

wrote:

Hi,

I do have same issue where log states unknown handler email, may I

know

the

need of having file.rb? I do not have this file in

/etc/sensu/handlers/,

do

I need to install anything or add in /etc/sensu/conf.d/ to get

started

in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson

wrote:

It looks like there is no handler configured with the name of

“default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check

“metrics”

response from the client while following the tutorial at link

Adding

a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12

1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#13

Can someone please share how to optimize graphite with sensu data?

I am a beginner here to use graphite with sensu to generate graphs for each checks. Please advise us with docs required to complete task in needful way.

···

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#14

Adding all other experts as well!

I would appreciate if anyone can help me in resolving this issue

.

···

On Wed, May 6, 2015 at 6:41 PM, Srinivas Palle srini5884@gmail.com wrote:

So far am using email handler to send me alerts, is this an issue causing me to disable communication between sensu and graphite? Do I need to use only TCP handler in sensu server to enable communication?

Please advise…

On Sun, May 3, 2015 at 11:42 PM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set up? Can we get history with in server?

Just trying to where do all the results gets stored before handling to handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#15

If I try to Configure Graphite’s carbon-cache for AMQP my sensu server stops working! .

···

On Wed, May 6, 2015 at 7:01 PM, Srinivas Palle srini5884@gmail.com wrote:

Adding all other experts as well!

I would appreciate if anyone can help me in resolving this issue

.

On Wed, May 6, 2015 at 6:41 PM, Srinivas Palle srini5884@gmail.com wrote:

So far am using email handler to send me alerts, is this an issue causing me to disable communication between sensu and graphite? Do I need to use only TCP handler in sensu server to enable communication?

Please advise…

On Sun, May 3, 2015 at 11:42 PM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set up? Can we get history with in server?

Just trying to where do all the results gets stored before handling to handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#16

DO I need to change handler in sensu server config.json file to amqp or tcp to make it work?

currently below is configuration I have in sensu server config file:

{

“rabbitmq”: {

"ssl": {

  "private_key_file": "/etc/sensu/ssl/client_key.pem",

  "cert_chain_file": "/etc/sensu/ssl/client_cert.pem"

},

"port": 5671,

"host": "localhost",

"user": "sensu",

"password": "sensu123",

"vhost": "/sensu"

},

“redis”: {

"host": "localhost",

"port": 6379

},

“api”: {

"host": "localhost",

"port": 4567

},

“dashboard”: {

"host": "localhost",

"port": 8080,

"user": "admin",

"password": "secret"

},

“handlers”: {

"default": {

  "type": "pipe",

  "command": "true"

}

}

}

Please let me know where exactly I need to make change to let sensu server to communicate with graphite? I am trying to send check results output directly to graphite over tcp or amqp handler.

···

On Wed, May 6, 2015 at 7:04 PM, Srinivas Palle srini5884@gmail.com wrote:

If I try to Configure Graphite’s carbon-cache for AMQP my sensu server stops working! .

On Wed, May 6, 2015 at 7:01 PM, Srinivas Palle srini5884@gmail.com wrote:

Adding all other experts as well!

I would appreciate if anyone can help me in resolving this issue

.

On Wed, May 6, 2015 at 6:41 PM, Srinivas Palle srini5884@gmail.com wrote:

So far am using email handler to send me alerts, is this an issue causing me to disable communication between sensu and graphite? Do I need to use only TCP handler in sensu server to enable communication?

Please advise…

On Sun, May 3, 2015 at 11:42 PM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set up? Can we get history with in server?

Just trying to where do all the results gets stored before handling to handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#17

tcp is probably the easiest way to send things to graphite from sensu
http://www.joemiller.me/2013/12/07/sensu-and-graphite-part-2/

No one can tell you exactly what you need to change because we don’t know about your needs.

Also, you may be confused about how tightly Sensu integrates with Graphite. Only graphite-aware plugins are going to send meaningful data the graphite handler. (like vmstat-metrics.rb)

Not all check results are going to be meaningful to graphite.

···

On Wed, May 6, 2015 at 8:58 AM, Srinivas Palle srini5884@gmail.com wrote:

DO I need to change handler in sensu server config.json file to amqp or tcp to make it work?

currently below is configuration I have in sensu server config file:

{

“rabbitmq”: {

"ssl": {
  "private_key_file": "/etc/sensu/ssl/client_key.pem",
  "cert_chain_file": "/etc/sensu/ssl/client_cert.pem"
},
"port": 5671,
"host": "localhost",
"user": "sensu",
"password": "sensu123",
"vhost": "/sensu"

},

“redis”: {

"host": "localhost",
"port": 6379

},

“api”: {

"host": "localhost",
"port": 4567

},

“dashboard”: {

"host": "localhost",
"port": 8080,
"user": "admin",
"password": "secret"

},

“handlers”: {

"default": {
  "type": "pipe",
  "command": "true"
}

}

}

Please let me know where exactly I need to make change to let sensu server to communicate with graphite? I am trying to send check results output directly to graphite over tcp or amqp handler.

On Wed, May 6, 2015 at 7:04 PM, Srinivas Palle srini5884@gmail.com wrote:

If I try to Configure Graphite’s carbon-cache for AMQP my sensu server stops working! .

On Wed, May 6, 2015 at 7:01 PM, Srinivas Palle srini5884@gmail.com wrote:

Adding all other experts as well!

I would appreciate if anyone can help me in resolving this issue

.

On Wed, May 6, 2015 at 6:41 PM, Srinivas Palle srini5884@gmail.com wrote:

So far am using email handler to send me alerts, is this an issue causing me to disable communication between sensu and graphite? Do I need to use only TCP handler in sensu server to enable communication?

Please advise…

On Sun, May 3, 2015 at 11:42 PM, Srinivas Palle srini5884@gmail.com wrote:

Thanks Kyle,

May I know where can we get history of events in sensu monitoring set up? Can we get history with in server?

Just trying to where do all the results gets stored before handling to handler.

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#18

Hi All,

Can someone please let me know if there any bugs with collectd.conf ? I do see lot of errors reporting the same on one particular but unfortunately none of the provided solutions are working.

Please help me how to resolve these issues?

  • Restarting statistics collection and monitoring daemon collectd Parse error in file /etc/collectd/collectd.conf', line 1100 near': syntax error, unexpected $end, expecting UNQUOTED_STRING or OPENBRAC or EOL

yyparse returned error #1

configfile: Cannot read file `/etc/collectd/collectd.conf’.

Unable to read config file /etc/collectd/collectd.conf.

Error: Reading the config file failed!

Read the syslog for details.

Thanks,

Srinivas

···

On Sun, May 3, 2015 at 11:16 PM, Kyle Anderson kyle@xkyle.com wrote:

Yes, you do need to install/deploy the handler script (mailer.rb, etc.

Usually in /etc/sensu/handlers/).

Additionally, you must add configuration for that handler

(/etc/sensu/conf.d/…) to configure that handler to perform actions on

events (to get alerts)

Read more about how to install and configure handlers at the docs page:

http://sensuapp.org/docs/0.17/getting-started-with-handlers

http://sensuapp.org/docs/0.17/handlers

On Sun, May 3, 2015 at 5:19 AM, Srinivas Palle srini5884@gmail.com wrote:

Hi,

I do have same issue where log states unknown handler email, may I know the

need of having file.rb? I do not have this file in /etc/sensu/handlers/, do

I need to install anything or add in /etc/sensu/conf.d/ to get started in

generating alerts from server?

Please advise me…

On Monday, March 2, 2015 at 12:10:40 AM UTC+5:30, Kyle Anderson wrote:

It looks like there is no handler configured with the name of “default”

http://sensuapp.org/docs/0.16/handlers

Good handlers for metrics are things like graphite, etc?

On Fri, Feb 27, 2015 at 3:28 PM, max2su...@gmail.com wrote:

Hi,

       We are facing a problem of not getting any check "metrics"

response from the client while following the tutorial at link Adding a

Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:"determining

stale clients"}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:"unknown

handler",“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:"publishing

check

request",“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:"publishing

check

result",“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:"Dev.load_avg.one

0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen

0.22 1425079537\n\n",“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#19

Hi Kyle,

My current sensu - graphite set up is working fine as expected with below exceptions, I ams till unable to send data through AMQP whenever I send data through AMQP carbon-cache gets crashed. Also please advice if below output is good to go with?

root@ip-192-168-30-160:/var/log/carbon# tail -f listener.log | grep -i “192.168.30.135”

03/06/2015 18:29:33 :: MetricLineReceiver connection with 192.168.30.135:47814 established

03/06/2015 18:29:33 :: MetricLineReceiver connection with 192.168.30.135:47814 closed cleanly

03/06/2015 18:39:44 :: MetricLineReceiver connection with 192.168.30.135:47813 lost: Connection to the other side was lost in a non-clean fashion: Connection lost.

AMQP error:

root@ip-192-168-30-160:~# service carbon-cache restart

  • Restarting Graphite backend daemon carbon-cache Traceback (most recent call last):

File “/usr/bin/carbon-cache”, line 32, in

run_twistd_plugin(__file__)

File “/usr/lib/python2.7/dist-packages/carbon/util.py”, line 90, in run_twistd_plugin

config.parseOptions(twistd_options)

File “/usr/lib/python2.7/dist-packages/twisted/application/app.py”, line 604, in parseOptions

usage.Options.parseOptions(self, options)

File “/usr/lib/python2.7/dist-packages/twisted/python/usage.py”, line 266, in parseOptions

self.subOptions.parseOptions(rest)

File “/usr/lib/python2.7/dist-packages/twisted/python/usage.py”, line 276, in parseOptions

self.postOptions()

File “/usr/lib/python2.7/dist-packages/carbon/conf.py”, line 197, in postOptions

program_settings = read_config(program, self)

File “/usr/lib/python2.7/dist-packages/carbon/conf.py”, line 526, in read_config

settings.readFrom(config, section)

File “/usr/lib/python2.7/dist-packages/carbon/conf.py”, line 146, in readFrom

value = parser.getboolean(section, key)

File “/usr/lib/python2.7/ConfigParser.py”, line 370, in getboolean

raise ValueError, 'Not a boolean: %s' % v

ValueError: Not a boolean: True

ENABLE_AMQP = True

AMQP_HOST = 192.168.30.135

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = sensu123

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

                               Thanks,Srini
···

On Saturday, February 28, 2015 at 4:58:56 AM UTC+5:30, max2su...@gmail.com wrote:

Hi,
We are facing a problem of not getting any check “metrics” response from the client while following the tutorial at link Adding a Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{“timestamp”:“2015-02-27T18:23:59.010129-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.362472-0500”,“level”:“info”,“message”:“determining stale clients”}

{“timestamp”:“2015-02-27T18:24:06.384124-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.386882-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:06.389126-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-27T18:24:08.426386-0500”,“level”:“info”,“message”:“publishing check request”,“payload”:{“name”:“load_metrics”,“issued”:1425079448,“command”:“load-metrics.rb”},“subscribers”:[“production”]}

Client > tail -f /var/log/sensu/sensu-client.log

{“timestamp”:“2015-02-27T18:25:37.674068-0500”,“level”:“info”,“message”:“publishing check result”,“payload”:{“client”:“sensu-client.25.max2”,“check”:{“name”:“load_metrics”,“issued”:1425079538,“command”:“load-metrics.rb”,“executed”:1425079537,“duration”:0.172,“output”:“Dev.load_avg.one 0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen 0.22 1425079537\n\n”,“status”:0}}}

What are the configs that we are missing ?

Thanks:-


#20

Well, "True" does look like a valid bool for python to me.
Can you put your carbon config file in a pastebin so I can see it
without email formatting?

···

On Wed, Jun 3, 2015 at 11:45 AM, Srinivas Palle <srini5884@gmail.com> wrote:

Hi Kyle,

My current sensu - graphite set up is working fine as expected with below
exceptions, I ams till unable to send data through AMQP whenever I send data
through AMQP carbon-cache gets crashed. Also please advice if below output
is good to go with?

root@ip-192-168-30-160:/var/log/carbon# tail -f listener.log | grep -i
"192.168.30.135"

03/06/2015 18:29:33 :: MetricLineReceiver connection with
192.168.30.135:47814 established

03/06/2015 18:29:33 :: MetricLineReceiver connection with
192.168.30.135:47814 closed cleanly

03/06/2015 18:39:44 :: MetricLineReceiver connection with
192.168.30.135:47813 lost: Connection to the other side was lost in a
non-clean fashion: Connection lost.

AMQP error:

root@ip-192-168-30-160:~# service carbon-cache restart

* Restarting Graphite backend daemon carbon-cache
Traceback (most recent call last):

  File "/usr/bin/carbon-cache", line 32, in <module>

    run_twistd_plugin(__file__)

  File "/usr/lib/python2.7/dist-packages/carbon/util.py", line 90, in
run_twistd_plugin

    config.parseOptions(twistd_options)

  File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line
604, in parseOptions

    usage.Options.parseOptions(self, options)

  File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 266,
in parseOptions

    self.subOptions.parseOptions(rest)

  File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 276,
in parseOptions

    self.postOptions()

  File "/usr/lib/python2.7/dist-packages/carbon/conf.py", line 197, in
postOptions

    program_settings = read_config(program, self)

  File "/usr/lib/python2.7/dist-packages/carbon/conf.py", line 526, in
read_config

    settings.readFrom(config, section)

  File "/usr/lib/python2.7/dist-packages/carbon/conf.py", line 146, in
readFrom

    value = parser.getboolean(section, key)

  File "/usr/lib/python2.7/ConfigParser.py", line 370, in getboolean

    raise ValueError, 'Not a boolean: %s' % v

ValueError: Not a boolean: True

ENABLE_AMQP = True

AMQP_HOST = 192.168.30.135

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = sensu123

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

                                   Thanks,Srini

On Saturday, February 28, 2015 at 4:58:56 AM UTC+5:30, max2su...@gmail.com > wrote:

Hi,
           We are facing a problem of not getting any check "metrics"
response from the client while following the tutorial at link Adding a
Metric.

Following are the outputs :-

Server > tail -f /var/log/sensu/sensu-server.log

{"timestamp":"2015-02-27T18:23:59.010129-0500","level":"error","message":"unknown
handler","handler_name":"default"}

{"timestamp":"2015-02-27T18:24:06.362472-0500","level":"info","message":"determining
stale clients"}

{"timestamp":"2015-02-27T18:24:06.384124-0500","level":"error","message":"unknown
handler","handler_name":"default"}

{"timestamp":"2015-02-27T18:24:06.386882-0500","level":"error","message":"unknown
handler","handler_name":"default"}

{"timestamp":"2015-02-27T18:24:06.389126-0500","level":"error","message":"unknown
handler","handler_name":"default"}

{"timestamp":"2015-02-27T18:24:08.426386-0500","level":"info","message":"publishing
check
request","payload":{"name":"load_metrics","issued":1425079448,"command":"load-metrics.rb"},"subscribers":["production"]}

Client > tail -f /var/log/sensu/sensu-client.log

{"timestamp":"2015-02-27T18:25:37.674068-0500","level":"info","message":"publishing
check
result","payload":{"client":"sensu-client.25.max2","check":{"name":"load_metrics","issued":1425079538,"command":"load-metrics.rb","executed":1425079537,"duration":0.172,"output":"Dev.load_avg.one
0.13 1425079537\nDev.load_avg.five 0.12 1425079537\nDev.load_avg.fifteen
0.22 1425079537\n\n","status":0}}}

What are the configs that we are missing ?

Thanks:-