Sensu backend not picking up subscriptions added to a host's agent.yml

Sensu Go
1

I have a host that had a subscriptions in its agent.yaml configuration. Sensu sees that subscription just fine.

I added two more subscriptions to the agent.yml file, and restarted the sensu agent. The sensu backend does not see these new subscriptions.

I deleted the host (entity) out of the sensu backend, and when I restart the agent on that host, the sensu backend can see the host again immediately, but it still only lists one of those subscriptions. What am I missing?

3

Yes, there are 2 checks in the subscription that Iā€™m trying to add. Sensuctl sees them. (One is a check that Iā€™m trying something new with - instead of running an asset, its just running a script on the host, like /opt/myscript.sh).

4

Tried deleting that weird check from the description (now the subscription only has a simple process check in it), and removing one of the subscriptions from the host, and removing the host from sensu and letting sensu rediscover it againā€¦still no dice.

6

So, calling my host <hostname>. Itā€™s in the namespace stg01. It is already subscribed to ā€˜adbā€™, Iā€™m trying to subscribe it to ā€˜postgresā€™ and ā€˜baseā€™ also, right now just focusing on the ā€˜postgresā€™ subscription.

agent.yml on <hostname>:

[alison.mccrea@<hostname> ]$ sudo cat /etc/sensu/agent.yml
---
backend-url:
- wss://<sensu_backend>:8081
subscriptions:
- base
- postgres
- adb
namespace: stg01

On the sensu backend:

[root@<sensu.backend> ~]$ sensuctl entity info <hostname> --namespace stg01
=== <hostname>
Name:                   <hostname>
Entity Class:           agent
Subscriptions:          adb, entity:<hostname>
Last Seen:              2020-06-03 19:33:26 +0000 UTC
Hostname:               <hostname>
OS:                     linux
Platform:               centos
Platform Family:        rhel
Platform Version:       6.10
Auto-Deregistration:    false
Deregistration Handler:

The check itself:

[root@<sensu.backend> ~]$ sensuctl check list --namespace stg01 | grep postgres
  postmaster_process            check-process.rb -p postmaster                                                      300                0     0   postgres        test_handler   sensu-ruby-runtime,sensu-plugins-process-checks           true       false
7

Nevermind. I found the problem. Itā€™s a problem Iā€™ve been having on some hosts for some reason ā€“

[alison.mccrea@<hostname>]$ sudo service sensu-agent restart
sensu-agent started
[alison.mccrea@<hostname>]$ ps aux | grep sensu
sensu     50512  0.0  0.9 127732 38328 ?        SNl  May11  31:37 /usr/sbin/sensu-agent start

The sensu-agent isnā€™t actually restarting, even though it says ā€œsensu-agent startedā€. Not actually sure why this happens but at least a real restart of the sensu agent fixed the issue.

1 Like
9

Thanks for the help anyway! Rubber duck debugging ftw

1 Like
10

Hi there. I recently ran into this same issue with a bunch of my entities. I started the agent with 3 subscriptions and after creating a check with new subscription (named zookeeper) and adding it to the agent, the new check was not picked up since the backend never recognized the new subscription in the entity. Initially when the agent was installed, the zookeeper subscription didnot exist but after setting up the new check with a new subscription and trying to add the subscription to the agent just didnot seem to pick this up. The timeline was as follows

  • Install agent with 3 subscriptions and see it on SensuUI
  • Add new check called check-zookeeper with new subscription zookeeper
  • Add new subscription to the agent.yml file and restart the sensu-agent with service sensu-agent restart
  • New subscription and checks not picked up by backend as per sensuctl and SensuUI
  • Delete the sensu-agent process and retry again but same failure
  • Stop the sensu-agent, wait for 5 minutes and then start again but same failure.

Agent verison:

[centos@zk-1 ~]$ sensu-agent version
sensu-agent version 6.2.0+ee, enterprise edition, build 62b5a984ace2698a52d590e732767a613b11d0b4, built 2020-12-16T23:33:11Z, built with go1.13.15

Agent configuration:

subscriptions:
  - webserver
  - all
  - system
  - zookeeper

backend-url:
  - "ws://<BACKEND>:8081"

name: "<HOSTNAME>"
namespace: "digital-services"
log-level: "info" # available log levels: panic, fatal, error, warn, info, debug

Check info using sensuctl check list | grep zookeeper :

[root@zk-1 centos]# sensuctl check list | grep zookeeper
  check-zookeeper   check-process.rb -p zookeeper                                                      60                0     0   zookeeper       opsgenie   sensu-ruby-runtime,process-plugin           true       false

Entity info using sensuctl entity info <HOSTNAME> :

=== <HOSTNAME>
Name:                   <HOSTNAME>
Entity Class:           agent
Subscriptions:          webserver, all, system, entity:<HOSTNAME>
Last Seen:              2021-01-09 17:31:47 +0000 UTC
Hostname:               <HOSTNAME>
OS:                     linux
Platform:               centos
Platform Family:        rhel
Platform Version:       7.8.2003
Auto-Deregistration:    false
Deregistration Handler:

Process information:

[root@zk-1 centos]# ps aux | grep sensu
sensu     8960  0.0  2.0 128564 40904 ?        Ssl  17:12   0:00 /usr/sbin/sensu-agent start
root     10910  0.0  0.1 112828  2316 pts/0    S+   17:34   0:00 grep --color=auto sensu

Any idea what went wrong in my approach? If it matters, the backend is running in a cluster mode with 3 instances