Hi,
I am reading the docs and testing Sensu, I am planning to use it with agents that have a spotty connection to the backend.
My question, while the connection to the backend is lost, will the agent continue running the checks it’s subscribed to using the information it was able to fetch the last time it connected to the backend?
Or will the checks stop completely until the connection is restored?
Hey,
No i don’t think the agent has a scheduler capacity. The backend is communicating all the scheduled check execution info to entities (also remember you can also execute checks in a just in time manner outside of the schedule)
The agent has some ability to queue up check results for dispatch back to the sensu backend but I believe in general the agent needs a websocket connection to a backend to recv each scheduled check requests.
Hi Javier,
The sensu-agent does not store any scheduler state by design. However, we’ve had other users in a similar position, and that lead us to develop a feature for sensu-agent that buffers events to disk until the agent can reconnect to the backend.
To make use of this feature, you’ll need to work out how to schedule the check yourself. The good news is that it can be as simple as a cron job.
The agent has a service running on port 3031 that you can send events to with HTTP POST, formatted in JSON. When the agent receives a valid event, it will queue it on disk before sending it to the backend. This way, the events can survive agent restarts, loss of backend connectivity, or other failure scenarios.
You can learn more in the docs, here: https://docs.sensu.io/sensu-go/latest/reference/agent/#create-monitoring-events-using-the-agent-api