alert filtering based on timezone


#1

Hi,

I’m trying to configure sensu handler through puppet which handles the alert only within business hours, like this

sensu handler for out of business hours

sensu::handler { ‘victorops_out’:
source => ‘puppet:///modules/server/sensu/handler-victorops.rb’,
command => ‘/etc/sensu/handlers/handler-victorops.rb’,
filters => [ ‘business_hours_only’ ],
severities => [ ‘critical’ ],
config => {
“api_url” => ‘https://alert.victorops.com/integrations/generic/20131114/alert/05e3de89-eb51-4ea5-825d-a2716e01af95’,
“routing_key” => ‘ops’
},
}

sensu filter for ‘victorops_out’ handler

sensu::filter { ‘business_hours_only’ :
ensure => “present”,
negate => false,
attributes => {
“check” => {
“interval” => 60
}
},
when => {
“days” => {
“all” => [
{
“begin” => “5:00 PM”,
“end” => “8:00 AM”
}
]
}
},
}

Problem here is, I’ve two node of different timezone one is AEST and other one is IST. So how can I filter out the notification time for each node ? Is there any way in sensu for accomplishing this ?


#2

Your servers should probably be set to UTC timezone to avoid this problem. Also filters are only evaluated on the sensu servers so having multiple sensu-client’s across different timezones should not be something you need to be concerned with. If you have two distinct sensu setups each one should have a filter to ensure that the appropriate team is alerted for their support hours I don’t think you really need two different filters on both setups.

To answer your question though its not possible with the when filter however it is possible using a custom filter leveraging the eval functionality:

{
  "filters": {
    "ten_to_ten_eastern": {
      "negate": false, # default: false
      "attributes": {
        "timestamp": "eval: ENV['TZ'] = 'America/New_York'; [1,2,3,4,5].include?(Time.at(value).wday) && Time.at(value).hour.between?(10,22)"
      }
    }
  }
}

Shameless advertising: I wrote a post that discusses filters: https://blog.sensu.io/alert-fatigue-part-2-alert-reduction-with-sensu-filters-token-substitution