# Changing handlers for a check not honored

**URL:** <https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475>\
**Category:** Sensu Classic (EOL)\
**Created:** [August 4, 2015, 8:34pm UTC](https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475 "2015-08-04T20:34:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ethan\_Erchinger](https://avatars.discourse-cdn.com/v4/letter/e/838e76/32.png) [@Ethan\_Erchinger](https://discourse.sensu.io/u/Ethan_Erchinger)\
**Post date:** [August 4, 2015, 8:34pm UTC](https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475/1 "2015-08-04T20:34:19Z")

</div>

Hi all,

I have recently changed all of our checks to use a different set of handlers, removing an older “hipchat” handler, and adding a new one. After applying the configuration and restarting the server, any new alerts seem to continue using the old set of handlers. The json listed here is the log entry on the master showing what it’s going to do with this check. The second block is the configuration block output on server startup, limited to this check.

`
{
“timestamp”: “2015-08-04T20:09:18.961191+0000”,
“level”: “info”,
“message”: “processing event”,
“event”: {
“id”: “ba5ba045-abf2-405b-acd2-7c5b84edb091”,
“client”: {
“name”: “host.erchinger.org”,
“address”: “10.208.22.113”,
“subscriptions”: [
“atd”
],
“version”: “0.19.2”,
“timestamp”: 1438718944
},
“check”: {
“command”: “/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1”,
“dependencies”: [
“sensu-server.erchinger.org/alerts-on_check”
],
“handlers”: [
“mailer”,
“hipchat”
],
“interval”: 60,
“occurrences”: 2,
“refresh”: 86400,
“subscribers”: [
“atd”
],
“name”: “atd_check”,
“issued”: 1438718958,
“executed”: 1438718958,
“duration”: 0.181,
“output”: “CheckProcs OK: Found 1 matching processes; cmd /atd/\n”,
“status”: 0,
“history”: [
“2”,
“2”,
“2”,
“0”
],
“total_state_change”: 11
},
“occurrences”: 3,
“action”: “resolve”,
“timestamp”: 1438718958
}
}
``
{ “atd_check”: [
null,
{
“command”: “/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1”,
“dependencies”: [
“sensu-server.erchinger.org/alerts-on_check”
],
“handlers”: [
“pagerduty”
],
“interval”: 60,
“occurrences”: 2,
“refresh”: 86400,
“subscribers”: [
“atd”
]
}
]}
`

Why wouldn’t the alert log entry shown first not adhere to the configuration read on startup? Do I have to clear something out of rabbitmq, or am I missing something else? For what it’s worth, I don’t even have anything with the word hipchat in the /etc/sensu directory anymore.

Thoughts?

Thanks,

Ethan

---

<div class="post-metadata">

**Author:** ![Ethan\_Erchinger](https://avatars.discourse-cdn.com/v4/letter/e/838e76/32.png) [@Ethan\_Erchinger](https://discourse.sensu.io/u/Ethan_Erchinger)\
**Post date:** [August 4, 2015, 8:36pm UTC](https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475/2 "2015-08-04T20:36:45Z")

</div>

Clearly the formatting into a code block didn’t work, for a reason I don’t understand. Anyway, reposted here.

{

“timestamp”: “2015-08-04T20:09:18.961191+0000”,

“level”: “info”,

“message”: “processing event”,

“event”: {

```
"id": "ba5ba045-abf2-405b-acd2-7c5b84edb091",

"client": {

  "name": "[host.erchinger.org](http://host.erchinger.org/)",

  "address": "10.208.22.113",

  "subscriptions": [

    "atd"

  ],

  "version": "0.19.2",

  "timestamp": 1438718944

},

"check": {

  "command": "/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1",

  "dependencies": [

    "[sensu-server.erchinger.org/alerts-on_check](http://sensu-server.erchinger.org/alerts-on_check)"

  ],

  "handlers": [

    "mailer",

    "hipchat"

  ],

  "interval": 60,

  "occurrences": 2,

  "refresh": 86400,

  "subscribers": [

    "atd"

  ],

  "name": "atd_check",

  "issued": 1438718958,

  "executed": 1438718958,

  "duration": 0.181,

  "output": "CheckProcs OK: Found 1 matching processes; cmd /atd/\n",

  "status": 0,

  "history": [

    "2",

    "2",

    "2",

    "0"

  ],

  "total_state_change": 11

},

"occurrences": 3,

"action": "resolve",

"timestamp": 1438718958

```

}

}

``

{“atd\_check”: [

null,

{

“command”: “/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1”,

“dependencies”: [

“[sensu-server.ord.1debit.com/alerts-on\_check](http://sensu-server.ord.1debit.com/alerts-on_check)”

],

“handlers”: [

“pagerduty”

],

“high\_flap\_threshold”: 25,

“interval”: 60,

“low\_flap\_threshold”: 10,

“occurrences”: 2,

“refresh”: 86400,

“subscribers”: [

“atd”

]

}

]}

``

---

<div class="post-metadata">

**Author:** ![Kyle\_Anderson](https://avatars.discourse-cdn.com/v4/letter/k/e5b9ba/32.png) [@Kyle\_Anderson](https://discourse.sensu.io/u/Kyle_Anderson)\
**Post date:** [August 5, 2015, 2:43am UTC](https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475/3 "2015-08-05T02:43:04Z")

</div>

Is atd\_check defined on host.erchinger.org at all? (the client)

If so, local configuration takes precedence:

> <https://github.com/sensu/sensu/blob/9a783db96ed350d35eb521875e467be01fc0d084/lib/sensu/client/process.rb#L196-L214>

If so, just adjust the local configuration and restart the sensu client.  
If not, can you paste client logs from host.erchinger.org for more context?

> **···**
>
> On Tue, Aug 4, 2015 at 1:36 PM, Ethan Erchinger \<ethan@erchinger.org\> wrote:
> 
> > Clearly the formatting into a code block didn't work, for a reason I don't  
> > understand. Anyway, reposted here.
> > 
> > {  
> > "timestamp": "2015-08-04T20:09:18.961191+0000",  
> > "level": "info",  
> > "message": "processing event",  
> > "event": {  
> > &nbsp;&nbsp;&nbsp;"id": "ba5ba045-abf2-405b-acd2-7c5b84edb091",  
> > &nbsp;&nbsp;&nbsp;"client": {  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"name": "host.erchinger.org",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"address": "10.208.22.113",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"subscriptions": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"atd"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"version": "0.19.2",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"timestamp": 1438718944  
> > &nbsp;&nbsp;&nbsp;},  
> > &nbsp;&nbsp;&nbsp;"check": {  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"command": "/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"dependencies": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"sensu-server.erchinger.org/alerts-on\_check"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"handlers": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"mailer",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"hipchat"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"interval": 60,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"occurrences": 2,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"refresh": 86400,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"subscribers": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"atd"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"name": "atd\_check",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"issued": 1438718958,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"executed": 1438718958,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"duration": 0.181,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"output": "CheckProcs OK: Found 1 matching processes; cmd /atd/\n",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"status": 0,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"history": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"2",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"2",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"2",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"0"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"total\_state\_change": 11  
> > &nbsp;&nbsp;&nbsp;},  
> > &nbsp;&nbsp;&nbsp;"occurrences": 3,  
> > &nbsp;&nbsp;&nbsp;"action": "resolve",  
> > &nbsp;&nbsp;&nbsp;"timestamp": 1438718958  
> > }  
> > }
> > 
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{"atd\_check": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;null,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"command": "/etc/sensu/plugins/processes/check-procs.rb -p atd -C  
> > 1",  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"dependencies": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"sensu-server.ord.1debit.com/alerts-on\_check"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"handlers": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"pagerduty"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"high\_flap\_threshold": 25,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"interval": 60,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"low\_flap\_threshold": 10,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"occurrences": 2,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"refresh": 86400,  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"subscribers": [  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"atd"  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;]  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;]}

---

<div class="post-metadata">

**Author:** ![Ethan\_Erchinger](https://avatars.discourse-cdn.com/v4/letter/e/838e76/32.png) [@Ethan\_Erchinger](https://discourse.sensu.io/u/Ethan_Erchinger)\
**Post date:** [August 11, 2015, 8:25pm UTC](https://discourse.sensu.io/t/changing-handlers-for-a-check-not-honored/475/4 "2015-08-11T20:25:29Z")

</div>

Just to wrap this up, I wasn’t using local configuration, only published checks. Things did finally start working as expected, it just took a couple hours for some reason.

Unfortunately one of the effected clients was destroyed by provisioning, so I’m not able to share the logs.

Ethan

> **···**
>
> On Tuesday, August 4, 2015 at 7:43:06 PM UTC-7, Kyle Anderson wrote:
> 
> > Is atd\_check defined on [host.erchinger.org](http://host.erchinger.org) at all? (the client)
> > 
> > If so, local configuration takes precedence:  
> > [https://github.com/sensu/sensu/blob/9a783db96ed350d35eb521875e467be01fc0d084/lib/sensu/client/process.rb#L196-L214](https://github.com/sensu/sensu/blob/9a783db96ed350d35eb521875e467be01fc0d084/lib/sensu/client/process.rb#L196-L214)
> > 
> > If so, just adjust the local configuration and restart the sensu client.  
> > If not, can you paste client logs from [host.erchinger.org](http://host.erchinger.org) for more context?
> > 
> > On Tue, Aug 4, 2015 at 1:36 PM, Ethan Erchinger [et...@erchinger.org](mailto:et...@erchinger.org) wrote:
> > 
> > > Clearly the formatting into a code block didn’t work, for a reason I don’t  
> > > understand. Anyway, reposted here.
> > > 
> > > {  
> > > “timestamp”: “2015-08-04T20:09:18.961191+0000”,  
> > > “level”: “info”,  
> > > “message”: “processing event”,  
> > > “event”: {  
> > > “id”: “ba5ba045-abf2-405b-acd2-7c5b84edb091”,  
> > > “client”: {  
> > > “name”: “[host.erchinger.org](http://host.erchinger.org)”,  
> > > “address”: “10.208.22.113”,  
> > > “subscriptions”: [  
> > > “atd”  
> > > ],  
> > > “version”: “0.19.2”,  
> > > “timestamp”: 1438718944  
> > > },  
> > > “check”: {  
> > > “command”: “/etc/sensu/plugins/processes/check-procs.rb -p atd -C 1”,  
> > > “dependencies”: [  
> > > “[sensu-server.erchinger.org/alerts-on\_check](http://sensu-server.erchinger.org/alerts-on_check)”  
> > > ],  
> > > “handlers”: [  
> > > “mailer”,  
> > > “hipchat”  
> > > ],  
> > > “interval”: 60,  
> > > “occurrences”: 2,  
> > > “refresh”: 86400,  
> > > “subscribers”: [  
> > > “atd”  
> > > ],  
> > > “name”: “atd\_check”,  
> > > “issued”: 1438718958,  
> > > “executed”: 1438718958,  
> > > “duration”: 0.181,  
> > > “output”: “CheckProcs OK: Found 1 matching processes; cmd /atd/\n”,  
> > > “status”: 0,  
> > > “history”: [  
> > > “2”,  
> > > “2”,  
> > > “2”,  
> > > “0”  
> > > ],  
> > > “total\_state\_change”: 11  
> > > },  
> > > “occurrences”: 3,  
> > > “action”: “resolve”,  
> > > “timestamp”: 1438718958  
> > > }  
> > > }
> > > 
> > > ```
> > > {"atd_check": [
> > > null,
> > > {
> > > "command": "/etc/sensu/plugins/processes/check-procs.rb -p atd -C
> > > 
> > > ```
> > > 
> > > 1",  
> > > “dependencies”: [  
> > > “[sensu-server.ord.1debit.com/alerts-on\_check](http://sensu-server.ord.1debit.com/alerts-on_check)”  
> > > ],  
> > > “handlers”: [  
> > > “pagerduty”  
> > > ],  
> > > “high\_flap\_threshold”: 25,  
> > > “interval”: 60,  
> > > “low\_flap\_threshold”: 10,  
> > > “occurrences”: 2,  
> > > “refresh”: 86400,  
> > > “subscribers”: [  
> > > “atd”  
> > > ]  
> > > }  
> > > ]}