When I install assets from Bonsai, where are the actual binaries that the agent execute during a check?
I am using Docker containers (image: sensu/sensu:latest) and although I can run checks just fine, I cannot see the binaries anywhere in the container or on the Docker host.
This is such a simple basic question, yet I cannot find that is being asked anywhere else.
Hmmmm I may have found them!!
In the Docker container, directories and binaries get put into place after creating a check:
ll /var/cache/sensu/sensu-agent/
total 64
drwxr–r-- 7 root root 4.0K May 24 17:37 .
drwxr–r-- 3 root root 25 May 20 14:04 …
drwxr-xr-x 3 root root 69 May 20 19:28 29a6e1725f37e533cd9f6a8a17b40ef757ba5ea5dc27701c19a7fb9ac9c2bdf9f684d758d4440f278bb7fc669e52bf02efb3fc6758731f7f3e10617a2fe026cf
drwxr-xr-x 5 root root 43 May 24 17:37 4b078df8694005d8908eea2903ae5194f8407240875af16804f1fbd55abf9aaeb8019588c2eddfb4fa545f2fdaa957d024146265b99ea3be2fa8e4579e014889
drwxr-xr-x 3 root root 69 May 21 22:43 612c6ff9928841090c4d23bf20aaf7558e4eed8977a848cf9e2899bb13a13e7540bac2b63e324f39d9b1257bb479676bc155b24e21bf93c722b812b0f15cb3bd
drwxr-xr-x 6 root root 56 May 20 14:05 8d768d1fba545898a8d09dca603457eb0018ec6829bc5f609a1ea51a2be0c4b2d13e1aa46139ecbb04873449e4c76f463f0bdfbaf2107caf37ab1c8db87d5250
drwxr-xr-x 5 root root 78 May 20 14:05 9dd717a7a71fa78b7647b27e9bde303ebf5b14dbf6a9aef6050d30cb6e255058b1c82d1bb9e5dfdcc2ce5ae8a0804964c807500142d09beb2cffb735d93c9797
-rw------- 1 root root 64.0K May 24 17:37 assets.db
-rw------- 1 root root 32.0K May 20 14:04 queue.db
Somehow, it appears that these directories and binaries get put into place magically on any node running an agent, i.e I ran the ‘sensuctl check create’ command on one agent, and all the other agents that are part of the same namespace have them as well.
Will this happen via the same Black Magic ™ on Windows nodes with agents installed?
Is there some general documention that speaks to these basic point on how Sensu works somewhere that I may have missed?
Thanks in advance!!
Sorry to hijack this thread, but I’ve installed Sensu Go per the documentation on CentOS 7 (no docker).
Then, following the documentation for Monitoring Server Resources, I tried installing the sensu-plugins-cpu-checks asset from Bonsai with sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks:4.1.0 -r cpu-checks-plugins, but I can’t find where the assets are.
Consequently, the check to run check-cpu.rb doesn’t complete because the file doesn’t actually exist…
I’ve executed find / -type f -name "check-cpu.rb" on the machine and it’s returned nothing, so the asset appears not to have downloaded properly.
What can I do to fix this?
Any and all thread hijacks are fine by me 
Could you please share the output of both
sensuctl asset list
sensuctl check list
Thanks for the quick reply!
sensuctl asset list:
Name URL Hash
──────────────────── ────────────────────────────────────────────────────────────────────────────────────────────── ─────────
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_debian9_linux_amd64.tar.gz 9dd717a
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_debian_linux_amd64.tar.gz f0435fd
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_centos7_linux_amd64.tar.gz 8a01862
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_centos6_linux_amd64.tar.gz f42be79
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_alpine3.8_linux_amd64.tar.gz 7a5ad2d
cpu-checks-plugins //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_alpine_linux_amd64.tar.gz a67676f
sensu-ruby-runtime //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_centos6_linux_amd64.tar.gz cbee191
sensu-ruby-runtime //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_debian_linux_amd64.tar.gz a28952f
sensu-ruby-runtime //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_centos_linux_amd64.tar.gz 338b88b
sensu-ruby-runtime //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_alpine_linux_amd64.tar.gz 8d768d1
sensuctl check list:
Name Command Interval Cron Timeout TTL Subscriptions Handlers Assets Hooks Publish? Stdin? Metric Format Metric Handlers
─────────── ────────────────────────── ────────── ────── ───────── ───── ─────────────── ────────── ─────────────────────────────────────── ─────── ────────── ──────── ─────────────── ─────────────────
check-cpu check-cpu.rb -w 75 -c 90 60 0 0 system cpu-checks-plugins,sensu-ruby-runtime true false
I figure it would also be useful to have the contents of my agent.yml:
--
# Sensu agent configuration
##
# agent overview
##
name: "[hostname redacted]"
namespace: "default"
subscriptions:
- system
#labels:
# example_key: "example value"
#annotations:
# example/key: "example value"
##
# agent configuration
##
backend-url:
- "ws://[hostname redacted]:8081"
cache-dir: "/var/cache/sensu/sensu-agent"
config-file: "/etc/sensu/agent.yml"
disable-assets: false
log-level: "debug" # available log levels: panic, fatal, error, warn, info, debug
##
# api configuration
##
api-host: "127.0.0.1"
api-port: 3031
disable-api: false
events-burst-limit: 10
events-rate-limit: 10.0
##
# authentication configuration
##
#user: "agent"
#password: "P@ssw0rd!"
##
# monitoring configuration
##
#deregister: false
#deregistration-handler: "example_handler"
#keepalive-warning-timeout: 120
#keepalive-interval: 20
##
# security configuration
##
#insecure-skip-tls-verify: false
#redact:
# - password
# - passwd
# - pass
# - api_key
# - api_token
# - access_key
# - secret_key
# - private_key
# - secret
#trusted-ca-file: "/path/to/trusted-certificate-authorities.pem"
##
# socket configuration
##
disable-sockets: false
socket-host: "127.0.0.1"
socket-port: 3030
##
# statsd configuration
##
#statsd-disable: false
#statsd-event-handlers:
# - example_handler
#statsd-flush-interval: 10
#statsd-metrics-host: "127.0.0.1"
#statsd-metrics-port: 8125
Sorry so long on a reply to your reply!
What type of host are you running the check against?
(What OS?)
Can you share the check error output as seen in the Dashboard under the check properties?
Hey,
Based on what you are reporting so far, its not clear that your check is even running or that your agent has established a connection with the backend. Before trying to troubleshoot if the assets are working, let’s first confirm that the check is running as scheduled and is creating a Sensu event.
Even if the asset configuration is botched, the check should be running as scheduled (at a 1 minute cadence), possibly returning an error. If the check isn’t running at all that suggests a couple of possible configuration problems that would prevent the agent from scheduling the check execution.
Double check that your sensuctl is configured to operate in the same namespace as your agent.
sensuctl config view
I doubt this is the problem since your agent is using default namespace right now and you are just starting to learn how to use Sensu. But if you have painted outside the lines a little bit while kicking the tires you might be in a different namespace than you thought so its good to confirm.
This is the more likely problem you may be having based on the information you have provided so far…
did you restart your sensu-agent service after changing the subscriptions to include system? Did the agent restart correctly? Is it configured to talk to the backend correctly?
You can ask the Sensu cluster what it knows about the registered entities to confirm the subscriptions are up2date.
sensuctl entity list
First is your agent in the list?
Second do the subscriptions look as you expect?
Third is the Last Seen time reasonable? (within 30 seconds or so with default agent settings)
I’ll show out part of my output as an example:
ID Class OS Subscriptions Last Seen
carbon-f31 agent linux test,linux, entity:carbon-f31 2020-07-06 15:21:27 -0800 AKDT
centos6-ipv6 agent linux linux,entity:centos6-ipv6 2020-07-06 15:21:16 -0800 AKDT
After confirming that the agent is participating in the system subscription using the entity list, I’d next make sure the cpu-check event has been created for the entity.
sensuctl event list
You should see an entry for a ‘keepalive’ check associated with your agent and it should have zero (OKAY) status. If the keepalive is missing or non-zero status, thats a problem with the agent connectivity with the backend we’ll need to sort out.
You should also see an entry in the output corresponding to your agent entity and the check-cpu check
If its not-zero status the output column should help us diagnose it.
It’s difficult to separate agent misconfiguration from some types of network misconfiguration so if anything above didn’t verify as expected, there are some networking issues to examine as well.
If you have a firewall or network security group in place, make sure the tcp 8081 port access is configured to let the agent in. Also be aware if you have some sort of proxy server in between managing the 8081 port, that its configured correctly for long lived websocket connections. If your proxy has problems with websockets, you should definitely see artifacts of connecting and reconnecting in your sensu-agent journal using journalctl.