Handlers and checks


#1

Hi Folks,

Couple of things.

  1. On a specific check based on the event which could be critical or warning I would like to send out email to different email distribution list. Secondly my current check configuration is for 60 seconds and I would not want emails to be triggered every second. I would like to trigger only once for a event and stop sending emails until its resolved. Is this possible?
  2. For sending out email I am using mail in linux environment.

{
“handlers”: {
“email”: {
“type”: “pipe”,
“command”: “mail -s ‘sensu alert’ your@address”
}
}
}


I am using mail command as I dont have smtp host to send emails. Would I be able to mail the event details in this scenario?

Thanks,

Mani

#2

You cannot use such a naive handler (mail binary) and achieve what you want.

You could start with the community mail handler:
https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the
community handlers respect:
http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:

There is currently no combination of tunables to do exactly what you
want, but you can modify the handler to have it's own filter_repeated
function with whatever logic you need.

···

On Wed, Sep 3, 2014 at 5:10 AM, reachmani <reachmani@gmail.com> wrote:

Hi Folks,

Couple of things.

On a specific check based on the event which could be critical or warning I
would like to send out email to different email distribution list. Secondly
my current check configuration is for 60 seconds and I would not want emails
to be triggered every second. I would like to trigger only once for a event
and stop sending emails until its resolved. Is this possible?
For sending out email I am using mail in linux environment.

{
  "handlers": {
    "email": {
      "type": "pipe",
      "command": "mail -s 'sensu alert' your@address"
    }
  }
}

I am using mail command as I dont have smtp host to send emails. Would I be
able to mail the event details in this scenario?

Thanks,
Mani


#3

Thanks for your response Kyle.

I have couple more follow up questions.

1> I have added refresh attribute in checks. But it doesn’t seems to be working, I still continue to get emails every minute. As per the documentation here is what it says.

Custom key-values can be added to a check definition, which will be included in event data, enabling handler creativity.

Common custom check definitions

occurrences: Number of event occurrences before the handler should take action

refresh: Number of seconds handlers should wait before taking second action. Relies on sensu-plugin.

Can you please let me know which plugin I need to install to get the refresh working. Where should I deploy this?

My sample check configuration

“check_disk”: {

“handlers”: [“handler_system_mail”],

“command”: “/opt/sensu/embedded/bin/ruby /etc/sensu/plugins/check-disk.rb”,

“interval”: 60,

“refresh”: 3600,

“subscribers”: [ “system” ]

}

2> Secondly, if I need to modify the existing handler code. Can you tell me the location and file name look like for modification?

Thanks,

Mani

···

On Wednesday, 3 September 2014 20:24:35 UTC+5:30, Kyle Anderson wrote:

You cannot use such a naive handler (mail binary) and achieve what you want.

You could start with the community mail handler:
https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the
community handlers respect:
http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:
https://github.com/sensu/sensu-plugin/blob/1c75c7f7f831f800395ba681ffe03de734a52ab7/lib/sensu-handler.rb#L82

There is currently no combination of tunables to do exactly what you
want, but you can modify the handler to have it’s own filter_repeated
function with whatever logic you need.

On Wed, Sep 3, 2014 at 5:10 AM, reachmani reac...@gmail.com wrote:

Hi Folks,

Couple of things.

On a specific check based on the event which could be critical or warning I
would like to send out email to different email distribution list. Secondly
my current check configuration is for 60 seconds and I would not want emails
to be triggered every second. I would like to trigger only once for a event
and stop sending emails until its resolved. Is this possible?
For sending out email I am using mail in linux environment.

{
“handlers”: {
“email”: {
“type”: “pipe”,
“command”: “mail -s ‘sensu alert’ your@address”
}
}
}

I am using mail command as I dont have smtp host to send emails. Would I be
able to mail the event details in this scenario?

Thanks,
Mani


#4

1. Are you using a handler that can interpret that data? (like most
from the community repo
https://github.com/sensu/sensu-community-plugins/tree/master/handlers/notification)
2. It is wherever you put it? Stock sensu comes with no handlers.
People (cfg mgmt tools) usually put them in /etc/sensu/handlers/

···

On Thu, Sep 4, 2014 at 3:09 AM, reachmani <reachmani@gmail.com> wrote:

Thanks for your response Kyle.

I have couple more follow up questions.

1> I have added refresh attribute in checks. But it doesn't seems to be
working, I still continue to get emails every minute. As per the
documentation here is what it says.

Custom key-values can be added to a check definition, which will be included
in event data, enabling handler creativity.

Common custom check definitions

occurrences: Number of event occurrences before the handler should take
action
refresh: Number of seconds handlers should wait before taking second action.
Relies on sensu-plugin.
Can you please let me know which plugin I need to install to get the refresh
working. Where should I deploy this?

My sample check configuration
"check_disk": {
      "handlers": ["handler_system_mail"],
      "command": "/opt/sensu/embedded/bin/ruby
/etc/sensu/plugins/check-disk.rb",
      "interval": 60,
      "refresh": 3600,
      "subscribers": [ "system" ]
    }

2> Secondly, if I need to modify the existing handler code. Can you tell me
the location and file name look like for modification?

Thanks,
Mani

On Wednesday, 3 September 2014 20:24:35 UTC+5:30, Kyle Anderson wrote:

You cannot use such a naive handler (mail binary) and achieve what you
want.

You could start with the community mail handler:

https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the
community handlers respect:
http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:

https://github.com/sensu/sensu-plugin/blob/1c75c7f7f831f800395ba681ffe03de734a52ab7/lib/sensu-handler.rb#L82

There is currently no combination of tunables to do exactly what you
want, but you can modify the handler to have it's own filter_repeated
function with whatever logic you need.

On Wed, Sep 3, 2014 at 5:10 AM, reachmani <reac...@gmail.com> wrote:
> Hi Folks,
>
> Couple of things.
>
> On a specific check based on the event which could be critical or
> warning I
> would like to send out email to different email distribution list.
> Secondly
> my current check configuration is for 60 seconds and I would not want
> emails
> to be triggered every second. I would like to trigger only once for a
> event
> and stop sending emails until its resolved. Is this possible?
> For sending out email I am using mail in linux environment.
>
> {
> "handlers": {
> "email": {
> "type": "pipe",
> "command": "mail -s 'sensu alert' your@address"
> }
> }
> }
>
> I am using mail command as I dont have smtp host to send emails. Would I
> be
> able to mail the event details in this scenario?
>
> Thanks,
> Mani


#5

I am using pipe handler, this could be the reason refresh isnt working?

“handler_system_mail”: {

“type”: “pipe”,

“command”: “echo ‘System critical!!!’”

},

···

On Thursday, 4 September 2014 20:22:59 UTC+5:30, Kyle Anderson wrote:

  1. Are you using a handler that can interpret that data? (like most

from the community repo

https://github.com/sensu/sensu-community-plugins/tree/master/handlers/notification)

  1. It is wherever you put it? Stock sensu comes with no handlers.

People (cfg mgmt tools) usually put them in /etc/sensu/handlers/

On Thu, Sep 4, 2014 at 3:09 AM, reachmani reac...@gmail.com wrote:

Thanks for your response Kyle.

I have couple more follow up questions.

1> I have added refresh attribute in checks. But it doesn’t seems to be

working, I still continue to get emails every minute. As per the

documentation here is what it says.

Custom key-values can be added to a check definition, which will be included

in event data, enabling handler creativity.

Common custom check definitions

occurrences: Number of event occurrences before the handler should take

action

refresh: Number of seconds handlers should wait before taking second action.

Relies on sensu-plugin.

Can you please let me know which plugin I need to install to get the refresh

working. Where should I deploy this?

My sample check configuration

“check_disk”: {

  "handlers": ["handler_system_mail"],
  "command": "/opt/sensu/embedded/bin/ruby

/etc/sensu/plugins/check-disk.rb",

  "interval": 60,
  "refresh": 3600,
  "subscribers": [ "system" ]
}

2> Secondly, if I need to modify the existing handler code. Can you tell me

the location and file name look like for modification?

Thanks,

Mani

On Wednesday, 3 September 2014 20:24:35 UTC+5:30, Kyle Anderson wrote:

You cannot use such a naive handler (mail binary) and achieve what you

want.

You could start with the community mail handler:

https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the

community handlers respect:

http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:

https://github.com/sensu/sensu-plugin/blob/1c75c7f7f831f800395ba681ffe03de734a52ab7/lib/sensu-handler.rb#L82

There is currently no combination of tunables to do exactly what you

want, but you can modify the handler to have it’s own filter_repeated

function with whatever logic you need.

On Wed, Sep 3, 2014 at 5:10 AM, reachmani reac...@gmail.com wrote:

Hi Folks,

Couple of things.

On a specific check based on the event which could be critical or

warning I

would like to send out email to different email distribution list.

Secondly

my current check configuration is for 60 seconds and I would not want

emails

to be triggered every second. I would like to trigger only once for a

event

and stop sending emails until its resolved. Is this possible?

For sending out email I am using mail in linux environment.

{

“handlers”: {

"email": {
  "type": "pipe",
  "command": "mail -s 'sensu alert' your@address"
}

}

}

I am using mail command as I dont have smtp host to send emails. Would I

be

able to mail the event details in this scenario?

Thanks,

Mani


#6

I am using mail command instead of echo in my code. Since my check executes ever minute my mail box gets filled. I am expecting refresh parameter to work so that I can set delay for my next notification.

···

On Thursday, 4 September 2014 21:35:44 UTC+5:30, reachmani wrote:

I am using pipe handler, this could be the reason refresh isnt working?

“handler_system_mail”: {

“type”: “pipe”,

“command”: “echo ‘System critical!!!’”

},

On Thursday, 4 September 2014 20:22:59 UTC+5:30, Kyle Anderson wrote:

  1. Are you using a handler that can interpret that data? (like most

from the community repo

https://github.com/sensu/sensu-community-plugins/tree/master/handlers/notification)

  1. It is wherever you put it? Stock sensu comes with no handlers.

People (cfg mgmt tools) usually put them in /etc/sensu/handlers/

On Thu, Sep 4, 2014 at 3:09 AM, reachmani reac...@gmail.com wrote:

Thanks for your response Kyle.

I have couple more follow up questions.

1> I have added refresh attribute in checks. But it doesn’t seems to be

working, I still continue to get emails every minute. As per the

documentation here is what it says.

Custom key-values can be added to a check definition, which will be included

in event data, enabling handler creativity.

Common custom check definitions

occurrences: Number of event occurrences before the handler should take

action

refresh: Number of seconds handlers should wait before taking second action.

Relies on sensu-plugin.

Can you please let me know which plugin I need to install to get the refresh

working. Where should I deploy this?

My sample check configuration

“check_disk”: {

  "handlers": ["handler_system_mail"],
  "command": "/opt/sensu/embedded/bin/ruby

/etc/sensu/plugins/check-disk.rb",

  "interval": 60,
  "refresh": 3600,
  "subscribers": [ "system" ]
}

2> Secondly, if I need to modify the existing handler code. Can you tell me

the location and file name look like for modification?

Thanks,

Mani

On Wednesday, 3 September 2014 20:24:35 UTC+5:30, Kyle Anderson wrote:

You cannot use such a naive handler (mail binary) and achieve what you

want.

You could start with the community mail handler:

https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the

community handlers respect:

http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:

https://github.com/sensu/sensu-plugin/blob/1c75c7f7f831f800395ba681ffe03de734a52ab7/lib/sensu-handler.rb#L82

There is currently no combination of tunables to do exactly what you

want, but you can modify the handler to have it’s own filter_repeated

function with whatever logic you need.

On Wed, Sep 3, 2014 at 5:10 AM, reachmani reac...@gmail.com wrote:

Hi Folks,

Couple of things.

On a specific check based on the event which could be critical or

warning I

would like to send out email to different email distribution list.

Secondly

my current check configuration is for 60 seconds and I would not want

emails

to be triggered every second. I would like to trigger only once for a

event

and stop sending emails until its resolved. Is this possible?

For sending out email I am using mail in linux environment.

{

“handlers”: {

"email": {
  "type": "pipe",
  "command": "mail -s 'sensu alert' your@address"
}

}

}

I am using mail command as I dont have smtp host to send emails. Would I

be

able to mail the event details in this scenario?

Thanks,

Mani


#7

Hi , I am trying to do the same thing (echoing to the console), but it does not works.
On running “sensu-server” , I get this in the logs sensu-server.log

{“timestamp”:“2015-02-25T11:33:32.799380-0500”,“level”:“error”,“message”:“unknown handler”,“handler_name”:“default”}

{“timestamp”:“2015-02-25T11:33:44.879601-0500”,“level”:“info”,“message”:“publishing check request”,“payload”:{“name”:“check_cpu”,“issued”:1424882024,“command”:“check-cpu.rb”},“subscribers”:[“production”]}

···

On Thursday, September 4, 2014 at 12:05:44 PM UTC-4, reachmani wrote:

I am using pipe handler, this could be the reason refresh isnt working?

“handler_system_mail”: {

“type”: “pipe”,

“command”: “echo ‘System critical!!!’”

},

On Thursday, 4 September 2014 20:22:59 UTC+5:30, Kyle Anderson wrote:

  1. Are you using a handler that can interpret that data? (like most
    from the community repo
    https://github.com/sensu/sensu-community-plugins/tree/master/handlers/notification)
  2. It is wherever you put it? Stock sensu comes with no handlers.
    People (cfg mgmt tools) usually put them in /etc/sensu/handlers/

On Thu, Sep 4, 2014 at 3:09 AM, reachmani reac...@gmail.com wrote:

Thanks for your response Kyle.

I have couple more follow up questions.

1> I have added refresh attribute in checks. But it doesn’t seems to be
working, I still continue to get emails every minute. As per the
documentation here is what it says.

Custom key-values can be added to a check definition, which will be included
in event data, enabling handler creativity.

Common custom check definitions

occurrences: Number of event occurrences before the handler should take
action
refresh: Number of seconds handlers should wait before taking second action.
Relies on sensu-plugin.
Can you please let me know which plugin I need to install to get the refresh
working. Where should I deploy this?

My sample check configuration
“check_disk”: {
“handlers”: [“handler_system_mail”],
“command”: “/opt/sensu/embedded/bin/ruby
/etc/sensu/plugins/check-disk.rb”,
“interval”: 60,
“refresh”: 3600,
“subscribers”: [ “system” ]
}

2> Secondly, if I need to modify the existing handler code. Can you tell me
the location and file name look like for modification?

Thanks,
Mani

On Wednesday, 3 September 2014 20:24:35 UTC+5:30, Kyle Anderson wrote:

You cannot use such a naive handler (mail binary) and achieve what you
want.

You could start with the community mail handler:

https://github.com/sensu/sensu-community-plugins/blob/master/handlers/notification/mailer.rb

Read the docs on some of the filter tunables that it most of the
community handlers respect:
http://sensuapp.org/docs/latest/checks#custom-check-definition-key-values

Here is the filter logic:

https://github.com/sensu/sensu-plugin/blob/1c75c7f7f831f800395ba681ffe03de734a52ab7/lib/sensu-handler.rb#L82

There is currently no combination of tunables to do exactly what you
want, but you can modify the handler to have it’s own filter_repeated
function with whatever logic you need.

On Wed, Sep 3, 2014 at 5:10 AM, reachmani reac...@gmail.com wrote:

Hi Folks,

Couple of things.

On a specific check based on the event which could be critical or
warning I
would like to send out email to different email distribution list.
Secondly
my current check configuration is for 60 seconds and I would not want
emails
to be triggered every second. I would like to trigger only once for a
event
and stop sending emails until its resolved. Is this possible?
For sending out email I am using mail in linux environment.

{
“handlers”: {
“email”: {
“type”: “pipe”,
“command”: “mail -s ‘sensu alert’ your@address”
}
}
}

I am using mail command as I dont have smtp host to send emails. Would I
be
able to mail the event details in this scenario?

Thanks,
Mani