Nagios has a spool directory where check results are written into text
files and queued for processing.
It is possible for results from multiple checks to be placed in a single
file, all of those results are then processed more efficiently by the
Nagios code.
ganglia-nagios-bridge[1] extracts all the metrics from a Ganglia gmetad
server in a single poll and dumps them all out to a single Nagios check
results file for bulk processing.
Can Sensu accept these Nagios bulk checkresult files?
If not, is there any alternative way to submit bulk check results from
tools like ganglia-nagios-bridge into Sensu? It has been observed
processing thousands of check results in one go, doing each of those
individually through the REST API[2] would appear to be very inefficient.
syslog-nagios-bridge[3] also uses this mechanism to submit results to
Nagios, but usually not with the same intensity as ganglia-nagios-bridge.
Regards,
Daniel
1. https://danielpocock.com/ganglia-nagios-bridge
2. https://sensuapp.org/docs/latest/api/results-api.html#results-post
3. https://github.com/dpocock/syslog-nagios-bridge
https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage
Is the socket you want to send bulk check results, but it consumes json of its own specification.
···
On Mon, Oct 17, 2016 at 7:14 AM, Daniel Pocock daniel@pocock.pro wrote:
Nagios has a spool directory where check results are written into text
files and queued for processing.
It is possible for results from multiple checks to be placed in a single
file, all of those results are then processed more efficiently by the
Nagios code.
ganglia-nagios-bridge[1] extracts all the metrics from a Ganglia gmetad
server in a single poll and dumps them all out to a single Nagios check
results file for bulk processing.
Can Sensu accept these Nagios bulk checkresult files?
If not, is there any alternative way to submit bulk check results from
tools like ganglia-nagios-bridge into Sensu? It has been observed
processing thousands of check results in one go, doing each of those
individually through the REST API[2] would appear to be very inefficient.
syslog-nagios-bridge[3] also uses this mechanism to submit results to
Nagios, but usually not with the same intensity as ganglia-nagios-bridge.
Regards,
Daniel
-
https://danielpocock.com/ganglia-nagios-bridge
-
https://sensuapp.org/docs/latest/api/results-api.html#results-post
-
https://github.com/dpocock/syslog-nagios-bridge
Can multiple results be submitted in a single TCP connection?
Or is it necessary to reconnect each time a result is submitted?
Doesn't the client then have to send them over RabbitMQ to the server,
is there no way to get thousands of check results directly into the
server process, bypassing the client and RabbitMQ?
···
On 17/10/16 16:43, Kyle Anderson wrote:
https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage
Is the socket you want to send bulk check results, but it consumes json
of its own specification.
Hmm. I’ve never tried submitting multiple requests to that socket at a time, but aparently you can, and even keep a consistent connection (but this doesn’t look like it is documented):
https://github.com/sensu/sensu/blob/899c298bca5e2c5dfa79a3622cb1b2ee59424f10/lib/sensu/client/socket.rb#L235-L238
But even via the API, it is still submitting to rabbitmq for the server to process.
I’m not aware of a method to bypass the transport. Maybe you could write an Extension. (Maybe the server could have a socket?
But if it did, how would you be sure you were talking to the current leader, etc?)
···
On Mon, Oct 17, 2016 at 7:46 AM, Daniel Pocock daniel@pocock.pro wrote:
On 17/10/16 16:43, Kyle Anderson wrote:
https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage
Is the socket you want to send bulk check results, but it consumes json
of its own specification.
Can multiple results be submitted in a single TCP connection?
Or is it necessary to reconnect each time a result is submitted?
Doesn’t the client then have to send them over RabbitMQ to the server,
is there no way to get thousands of check results directly into the
server process, bypassing the client and RabbitMQ?
Las time I checked, you can't. The tcp socket is read as long as the data
received fails to parse as a json string or a timeout occurrs (500msec
iirc). If you try to submit multiple check results, how would you append
one check result to another? json parsing would fail unless you sent a json
array of check results, in which case code will fail cuz is not expecting
an array, but just a json object (hash/dictionary).
···
On Mon, Oct 17, 2016 at 10:59 AM, Kyle Anderson <kyle@xkyle.com> wrote:
Hmm. I've never tried submitting multiple requests to that socket at a
time, but aparently you can, and even keep a consistent connection (but
this doesn't look like it is documented):
GitHub - sensu/sensu at 899c298bca5e2c5dfa79a3622cb1b2ee59424f10
ee59424f10/lib/sensu/client/socket.rb#L235-L238
aside: transport (rabbitmq usually) is the servers work queue. you don’t want to bypass it.
···
On Oct 17, 2016, at 7:46 AM, Daniel Pocock <daniel@pocock.pro> wrote:
is there no way to get thousands of check results directly into the
server process, bypassing the client and RabbitMQ?
Would you consider supporting some bulk imports through a dedicated
queue in either RabbitMQ or directly into Redis perhaps? I notice you
also support Redis as a transport now, can both Redis and RabbitMQ
transports be enabled concurrently for different clients?
Something like ganglia-nagios-bridge (or ganglia-sensu-bridge) would
probably run on the same host as the Sensu server itself.
Regards,
Daniel
···
On 17/10/16 19:25, John Fessenden wrote:
aside: transport (rabbitmq usually) is the servers work queue. you don’t want to bypass it.