Where are my assets if I added from Bonsai?

When I install assets from Bonsai, where are the actual binaries that the agent execute during a check?
I am using Docker containers (image: sensu/sensu:latest) and although I can run checks just fine, I cannot see the binaries anywhere in the container or on the Docker host.

This is such a simple basic question, yet I cannot find that is being asked anywhere else.

Hmmmm I may have found them!!
In the Docker container, directories and binaries get put into place after creating a check:

ll /var/cache/sensu/sensu-agent/
total 64
drwxr–r-- 7 root root 4.0K May 24 17:37 .
drwxr–r-- 3 root root 25 May 20 14:04 …
drwxr-xr-x 3 root root 69 May 20 19:28 29a6e1725f37e533cd9f6a8a17b40ef757ba5ea5dc27701c19a7fb9ac9c2bdf9f684d758d4440f278bb7fc669e52bf02efb3fc6758731f7f3e10617a2fe026cf
drwxr-xr-x 5 root root 43 May 24 17:37 4b078df8694005d8908eea2903ae5194f8407240875af16804f1fbd55abf9aaeb8019588c2eddfb4fa545f2fdaa957d024146265b99ea3be2fa8e4579e014889
drwxr-xr-x 3 root root 69 May 21 22:43 612c6ff9928841090c4d23bf20aaf7558e4eed8977a848cf9e2899bb13a13e7540bac2b63e324f39d9b1257bb479676bc155b24e21bf93c722b812b0f15cb3bd
drwxr-xr-x 6 root root 56 May 20 14:05 8d768d1fba545898a8d09dca603457eb0018ec6829bc5f609a1ea51a2be0c4b2d13e1aa46139ecbb04873449e4c76f463f0bdfbaf2107caf37ab1c8db87d5250
drwxr-xr-x 5 root root 78 May 20 14:05 9dd717a7a71fa78b7647b27e9bde303ebf5b14dbf6a9aef6050d30cb6e255058b1c82d1bb9e5dfdcc2ce5ae8a0804964c807500142d09beb2cffb735d93c9797
-rw------- 1 root root 64.0K May 24 17:37 assets.db
-rw------- 1 root root 32.0K May 20 14:04 queue.db

Somehow, it appears that these directories and binaries get put into place magically on any node running an agent, i.e I ran the β€˜sensuctl check create’ command on one agent, and all the other agents that are part of the same namespace have them as well.

Will this happen via the same Black Magic β„’ on Windows nodes with agents installed?

Is there some general documention that speaks to these basic point on how Sensu works somewhere that I may have missed?

Thanks in advance!!

A lot of this information is contained in the Assets reference docs.

1 Like

Sorry to hijack this thread, but I’ve installed Sensu Go per the documentation on CentOS 7 (no docker).

Then, following the documentation for Monitoring Server Resources, I tried installing the sensu-plugins-cpu-checks asset from Bonsai with sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks:4.1.0 -r cpu-checks-plugins, but I can’t find where the assets are.

Consequently, the check to run check-cpu.rb doesn’t complete because the file doesn’t actually exist…

I’ve executed find / -type f -name "check-cpu.rb" on the machine and it’s returned nothing, so the asset appears not to have downloaded properly.

What can I do to fix this?

Any and all thread hijacks are fine by me :slight_smile:
Could you please share the output of both
sensuctl asset list
sensuctl check list

Thanks for the quick reply!

sensuctl asset list:

         Name                                                      URL                                                 Hash
 ──────────────────── ────────────────────────────────────────────────────────────────────────────────────────────── ─────────
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_debian9_linux_amd64.tar.gz         9dd717a
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_debian_linux_amd64.tar.gz          f0435fd
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_centos7_linux_amd64.tar.gz         8a01862
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_centos6_linux_amd64.tar.gz         f42be79
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_alpine3.8_linux_amd64.tar.gz       7a5ad2d
  cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_alpine_linux_amd64.tar.gz          a67676f
  sensu-ruby-runtime   //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_centos6_linux_amd64.tar.gz   cbee191
  sensu-ruby-runtime   //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_debian_linux_amd64.tar.gz    a28952f
  sensu-ruby-runtime   //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_centos_linux_amd64.tar.gz    338b88b
  sensu-ruby-runtime   //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_alpine_linux_amd64.tar.gz    8d768d1

sensuctl check list:

    Name              Command            Interval   Cron   Timeout   TTL   Subscriptions   Handlers                  Assets                   Hooks   Publish?   Stdin?   Metric Format   Metric Handlers
 ─────────── ────────────────────────── ────────── ────── ───────── ───── ─────────────── ────────── ─────────────────────────────────────── ─────── ────────── ──────── ─────────────── ─────────────────
  check-cpu   check-cpu.rb -w 75 -c 90         60                0     0   system                     cpu-checks-plugins,sensu-ruby-runtime           true       false

I figure it would also be useful to have the contents of my agent.yml:

# Sensu agent configuration

# agent overview
name: "[hostname redacted]"
namespace: "default"
  - system
#  example_key: "example value"
#  example/key: "example value"

# agent configuration
  - "ws://[hostname redacted]:8081"
cache-dir: "/var/cache/sensu/sensu-agent"
config-file: "/etc/sensu/agent.yml"
disable-assets: false
log-level: "debug" # available log levels: panic, fatal, error, warn, info, debug

# api configuration
api-host: ""
api-port: 3031
disable-api: false
events-burst-limit: 10
events-rate-limit: 10.0

# authentication configuration
#user: "agent"
#password: "P@ssw0rd!"

# monitoring configuration
#deregister: false
#deregistration-handler: "example_handler"
#keepalive-warning-timeout: 120
#keepalive-interval: 20

# security configuration
#insecure-skip-tls-verify: false
#  - password
#  - passwd
#  - pass
#  - api_key
#  - api_token
#  - access_key
#  - secret_key
#  - private_key
#  - secret
#trusted-ca-file: "/path/to/trusted-certificate-authorities.pem"

# socket configuration
disable-sockets: false
socket-host: ""
socket-port: 3030

# statsd configuration
#statsd-disable: false
#  - example_handler
#statsd-flush-interval: 10
#statsd-metrics-host: ""
#statsd-metrics-port: 8125

Sorry so long on a reply to your reply!
What type of host are you running the check against?
(What OS?)
Can you share the check error output as seen in the Dashboard under the check properties?

Based on what you are reporting so far, its not clear that your check is even running or that your agent has established a connection with the backend. Before trying to troubleshoot if the assets are working, let’s first confirm that the check is running as scheduled and is creating a Sensu event.

Even if the asset configuration is botched, the check should be running as scheduled (at a 1 minute cadence), possibly returning an error. If the check isn’t running at all that suggests a couple of possible configuration problems that would prevent the agent from scheduling the check execution.

1. Your agent and your check maybe in different Sensu namespaces

Double check that your sensuctl is configured to operate in the same namespace as your agent.

sensuctl config view

I doubt this is the problem since your agent is using default namespace right now and you are just starting to learn how to use Sensu. But if you have painted outside the lines a little bit while kicking the tires you might be in a different namespace than you thought so its good to confirm.

2. Agent may be misconfigured or needs to be restarted

This is the more likely problem you may be having based on the information you have provided so far…
did you restart your sensu-agent service after changing the subscriptions to include system? Did the agent restart correctly? Is it configured to talk to the backend correctly?

You can ask the Sensu cluster what it knows about the registered entities to confirm the subscriptions are up2date.

sensuctl entity list

First is your agent in the list?
Second do the subscriptions look as you expect?
Third is the Last Seen time reasonable? (within 30 seconds or so with default agent settings)

I’ll show out part of my output as an example:

      ID        Class    OS                         Subscriptions                                 Last Seen             
  carbon-f31     agent   linux   test,linux, entity:carbon-f31   2020-07-06 15:21:27 -0800 AKDT  
  centos6-ipv6   agent   linux   linux,entity:centos6-ipv6       2020-07-06 15:21:16 -0800 AKDT  

After confirming that the agent is participating in the system subscription using the entity list, I’d next make sure the cpu-check event has been created for the entity.

sensuctl event list

You should see an entry for a β€˜keepalive’ check associated with your agent and it should have zero (OKAY) status. If the keepalive is missing or non-zero status, thats a problem with the agent connectivity with the backend we’ll need to sort out.

You should also see an entry in the output corresponding to your agent entity and the check-cpu check
If its not-zero status the output column should help us diagnose it.

3. Network misconfiguration

It’s difficult to separate agent misconfiguration from some types of network misconfiguration so if anything above didn’t verify as expected, there are some networking issues to examine as well.

If you have a firewall or network security group in place, make sure the tcp 8081 port access is configured to let the agent in. Also be aware if you have some sort of proxy server in between managing the 8081 port, that its configured correctly for long lived websocket connections. If your proxy has problems with websockets, you should definitely see artifacts of connecting and reconnecting in your sensu-agent journal using journalctl.