Having trouble generating sensu-agent logs on Windows Server 2019

Sensu Go
1

According to the logs, the log files on Windows should be found at C:\ProgramData\sensu\log\sensu-agent.log.

Simply put, the logs are not being outputted. I tried manually specifying the log file location by setting log-file to C:\ProgramData\sensu\log\sensu-agent.log, but that seemed to have no effect either.

The sensu-agent version is

sensu-agent version 6.7.2+ee, enterprise edition, build e0d4063f2671a83ef64938c0ac7d2f3bd30e0a8d, built 2022-05-11T21:42:12Z, built with go1.17.6

And it is running on Windows Server 2019.

The associated yaml file looks something like this (edited to remove some sensitive informational)

name: "Agent_Name"
namespace: "staging"
subscriptions:
 - system
backend-url:
  - "wss://monitoring.example.com:8081"
cache-dir: 'C:\ProgramData\sensu\cache\sensu-agent'
log-level: "info"
log-file: 'C:\ProgramData\sensu\log\sensu-agent.log'
api-host: "0.0.0.0"
api-port: 3031
keepalive-pipelines:
- core/v2.Pipeline.email-alerts
trusted-ca-file: 'C:\ProgramData\sensu\certs\ca.pem'
cert-file: 'C:\ProgramData\sensu\certs\agent.pem'
key-file: 'C:\ProgramData\sensu\certs\agent-key.pem'
2

I just tried upgrading to 6.7.4 but it didn’t fix my problem.

3

I set my log level to debug, and then executed sensu-agent in powershell.

Here is the output from the first few seconds.

{"component":"agent","level":"info","msg":"compacting api queue","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"finished api queue compaction","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"using tls client auth","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating agent name","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating keepalive warning timeout","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating keepalive critical timeout","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating backend URLs is defined","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating backend URLs: [wss://monitoring.example.com:8081/]","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating backend URL: wss://monitoring.example.com:8081/","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"validating keepalive pipelines: [core/v2.Pipeline.email-alerts]","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"configuration successfully validated","time":"2022-07-25T10:24:28+09:00"}
{"cache":"C:\\ProgramData\\sensu\\cache\\sensu-agent","component":"asset-manager","level":"debug","msg":"initializing cache directory","time":"2022-07-25T10:24:28+09:00"}
{"cache":"C:\\ProgramData\\sensu\\cache\\sensu-agent","component":"asset-manager","level":"debug","msg":"done initializing cache directory","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"starting statsd server on address: 127.0.0.1:8125","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"starting api on address: 0.0.0.0:3031","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"starting UDP listener on address: 127.0.0.1:3030","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"starting TCP listener on address: 127.0.0.1:3030","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","error":"listen udp 127.0.0.1:8125: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.","level":"error","msg":"statsd listener failed on 127.0.0.1:8125","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"connecting to backend URL \"wss://monitoring.example.com:8081/\"","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","header":"Accept: application/octet-stream","level":"debug","msg":"setting header","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"info","msg":"successfully connected","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","header":"Accept: [application/octet-stream application/json]","level":"debug","msg":"received header","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","format":"protobuf","level":"debug","msg":"setting serialization/deserialization","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","header":"Content-Type: application/octet-stream","level":"debug","msg":"setting header","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"maxSessionLength is 0s, agent won't periodically disconnect","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","content_type":"application/octet-stream","level":"info","msg":"message received","payload_size":175,"time":"2022-07-25T10:24:28+09:00","type":"entity_config"}
{"component":"agent","level":"debug","msg":"received an entity config from the backend","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","level":"debug","msg":"successfully received the initial entity config","time":"2022-07-25T10:24:28+09:00"}
{"check":"keepalive","component":"agent","entity":"Agent_Name","event_uuid":"2335727a-e6ba-4f9a-af01-05668333be3c","level":"info","msg":"sending event to backend","time":"2022-07-25T10:24:28+09:00"}
{"component":"agent","content_type":"application/octet-stream","level":"info","msg":"message received","payload_size":2647,"time":"2022-07-25T10:24:29+09:00","type":"check_request"}
{"component":"agent","level":"info","msg":"scheduling check execution: cpu-check","time":"2022-07-25T10:24:29+09:00"}
{"assets":["sensu/check-cpu-usage"],"check":"cpu-check","component":"agent","level":"debug","msg":"fetching assets for check","namespace":"staging","time":"2022-07-25T10:24:29+09:00"}
{"asset":"sensu/check-cpu-usage","component":"asset-manager","entity":"Agent_Name","level":"info","msg":"asset includes builds, using builds instead of asset","time":"2022-07-25T10:24:29+09:00"}
{"asset":"sensu/check-cpu-usage","component":"asset-manager","entity":"Agent_Name","filter":["entity.system.os == 'windows'","entity.system.arch == 'amd64'"],"level":"debug","msg":"entity filtered, installing asset build","time":"2022-07-25T10:24:29+09:00"}
{"check":"cpu-check","component":"agent","entity":"Agent_Name","event_uuid":"24b0025d-147b-4a54-9527-d1154a03767f","level":"info","metrics":true,"msg":"sending event to backend","time":"2022-07-25T10:24:31+09:00"}
{"component":"agent","content_type":"application/octet-stream","level":"info","msg":"sending message","payload_size":1285,"time":"2022-07-25T10:24:31+09:00","type":"event"}
{"component":"agent","content_type":"application/octet-stream","level":"info","msg":"message received","payload_size":2647,"time":"2022-07-25T10:24:39+09:00","type":"check_request"}
{"component":"agent","level":"info","msg":"scheduling check execution: cpu-check","time":"2022-07-25T10:24:39+09:00"}
{"assets":["sensu/check-cpu-usage"],"check":"cpu-check","component":"agent","level":"debug","msg":"fetching assets for check","namespace":"staging","time":"2022-07-25T10:24:39+09:00"}
{"asset":"sensu/check-cpu-usage","component":"asset-manager","entity":"Agent_Name","level":"info","msg":"asset includes builds, using builds instead of asset","time":"2022-07-25T10:24:39+09:00"}
{"asset":"sensu/check-cpu-usage","component":"asset-manager","entity":"Agent_Name","filter":["entity.system.os == 'windows'","entity.system.arch == 'amd64'"],"level":"debug","msg":"entity filtered, installing asset build","time":"2022-07-25T10:24:39+09:00"}
{"check":"cpu-check","component":"agent","entity":"Agent_Name","event_uuid":"12bc2ad9-d0d0-41d8-bd4c-d80dea514fe7","level":"info","metrics":true,"msg":"sending event to backend","time":"2022-07-25T10:24:41+09:00"}
{"component":"agent","content_type":"application/octet-stream","level":"info","msg":"sending message","payload_size":1285,"time":"2022-07-25T10:24:41+09:00","type":"event"}
{"component":"agent","level":"warning","msg":"signal \"interrupt\" received, shutting down agent","time":"2022-07-25T10:24:46+09:00"}
{"component":"transport","level":"debug","msg":"websocket connection has been closed, stopping the heartbeat","time":"2022-07-25T10:24:46+09:00"}
4

I figured it out!

If you run the sensu-agent directly, for example by executing sensu-agent.exe in Powershell, it does not produce a log. This is also true for the linux version of the Sensu Agent.

Instead, it needs to be installed and run as a service. So in Windows, that means running sensu-agent service install.

After that, the logging works as expected.

In my case, I was using Windows Task Scheduler to run sensu-agent.exe, which is why I wasn’t getting any log output.

5

Hi Michael :wave:

Glad you were able to figure this out and thanks for posting your solution!