Subscription check flags bleeding over into other subscriptions


#1

I have run into an issue where I have a check such as one that checks for numbers of processors on the node that would have a general syntax such as the following for a subscription.
cat conf.d/check-ivy.json

“check-corecount”: {
“handlers”: [“default”, “mailer”],
“command”: “/etc/sensu/plugins/common/check-corecount.sh -w 18 -c 10”,
“interval”: 60,
“subscribers”: [ “ivy” ]
},

Where as we would also have one for some sandy bridge nodes

“check-corecount”: {
“handlers”: [“default”, “mailer”],
“command”: “/etc/sensu/plugins/common/check-corecount.sh -w 15 -c 8”,
“interval”: 60,
“subscribers”: [ “sandy” ]
},

However any node that is subscribed to the sandy subscription also gets the ivy subscription and of course the ivy flags for the check-corecount.sh script.

This hasn’t been an issue in the past few months that sensu has been running and was wondering if this is expected behaviour from sensu or perhaps a symptom of something else.

Thanks,
JB


#2

It surprises me that the server would allow you to define two checks
with the same name and it would work.
Sensu loads up all the checks into one big hash.

The only other thing that reminds me of this is:

···

On Fri, Oct 31, 2014 at 12:51 PM, John B <wilshire461@gmail.com> wrote:

I have run into an issue where I have a check such as one that checks for
numbers of processors on the node that would have a general syntax such as
the following for a subscription.
cat conf.d/check-ivy.json
...
    "check-corecount": {
      "handlers": ["default", "mailer"],
      "command": "/etc/sensu/plugins/common/check-corecount.sh -w 18 -c 10",
      "interval": 60,
      "subscribers": [ "ivy" ]
      },
...

Where as we would also have one for some sandy bridge nodes
...
    "check-corecount": {
      "handlers": ["default", "mailer"],
      "command": "/etc/sensu/plugins/common/check-corecount.sh -w 15 -c 8",
      "interval": 60,
      "subscribers": [ "sandy" ]
      },
...

However any node that is subscribed to the sandy subscription also gets the
ivy subscription and of course the ivy flags for the check-corecount.sh
script.

This hasn't been an issue in the past few months that sensu has been running
and was wondering if this is expected behaviour from sensu or perhaps a
symptom of something else.

Thanks,
JB


#3

I was thinking it in the way that why would you bother to have two exact same checks that do the exact same thing defined twice.

I read into a few other issues connected with #806 and it looks as though just defining some ruby variables that then feed into the check should work? Seems very silly to have multiple copies of the same check that one has for each and every subscription.

···

On Friday, October 31, 2014 10:42:13 PM UTC-5, Kyle Anderson wrote:

It surprises me that the server would allow you to define two checks

with the same name and it would work.

Sensu loads up all the checks into one big hash.

The only other thing that reminds me of this is:

https://github.com/sensu/sensu/issues/806#issuecomment-60816339


#4

Sensu-server is doing a big merge on all the json to load in the config.

Then, for each check it goes through the hash and schedules them:

When it schedules "check-corecount".... what will the contents of the
hash be? I'm pretty sure it will be the merge of both, the last file
read will win.

It doesn't seem silly to me to have multiple copies of this check, you
need check-corecount-sandy and check-corecount-ivy.
It's not about having different copies for each subscription, it is
about having different copies for each *config*.

Personally, I like the standalone module, check-corecount could be
deployed client-side to your nodes, and they would have different
definitions depending on the client. (as opposed to the
publish/subscribe module)

···

On Sun, Nov 2, 2014 at 1:53 PM, John B <wilshire461@gmail.com> wrote:

I was thinking it in the way that why would you bother to have two exact
same checks that do the exact same thing defined twice.

I read into a few other issues connected with #806 and it looks as though
just defining some ruby variables that then feed into the check should work?
Seems very silly to have multiple copies of the same check that one has for
each and every subscription.

On Friday, October 31, 2014 10:42:13 PM UTC-5, Kyle Anderson wrote:

It surprises me that the server would allow you to define two checks
with the same name and it would work.
Sensu loads up all the checks into one big hash.

The only other thing that reminds me of this is:
https://github.com/sensu/sensu/issues/806#issuecomment-60816339


#5

https://github.com/sensu/sensu-settings/blob/95fcb0aa217a10a5deb007cdf05eec8273e5aee0/lib/sensu/settings/loader.rb#L91

Then, for each check it goes through the hash and schedules them:

https://github.com/sensu/sensu/blob/cb7780f268ee3b74d06156e0481ed1e4c7d1e7db/lib/sensu/server.rb#L512

When it schedules “check-corecount”… what will the contents of the

hash be? I’m pretty sure it will be the merge of both, the last file

read will win.

Sensu-server is doing a big merge on all the json to load in the config.

Thanks for the source references.

It doesn’t seem silly to me to have multiple copies of this check, you

need check-corecount-sandy and check-corecount-ivy.

It’s not about having different copies for each subscription, it is

about having different copies for each config.

Personally, I like the standalone module, check-corecount could be

deployed client-side to your nodes, and they would have different

definitions depending on the client. (as opposed to the

publish/subscribe module)

I forgot to even think about the standalone check path. Thank you for all the help!