# bulk check result submission into Sensu

**URL:** <https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700>\
**Category:** Sensu Classic (EOL)\
**Created:** [October 17, 2016, 2:14pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700 "2016-10-17T14:14:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Pocock](https://avatars.discourse-cdn.com/v4/letter/d/8e8cbc/32.png) [@Daniel\_Pocock](https://discourse.sensu.io/u/Daniel_Pocock)\
**Post date:** [October 17, 2016, 2:14pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/1 "2016-10-17T14:14:48Z")

</div>

Nagios has a spool directory where check results are written into text  
files and queued for processing.

It is possible for results from multiple checks to be placed in a single  
file, all of those results are then processed more efficiently by the  
Nagios code.

ganglia-nagios-bridge[1] extracts all the metrics from a Ganglia gmetad  
server in a single poll and dumps them all out to a single Nagios check  
results file for bulk processing.

Can Sensu accept these Nagios bulk checkresult files?

If not, is there any alternative way to submit bulk check results from  
tools like ganglia-nagios-bridge into Sensu? It has been observed  
processing thousands of check results in one go, doing each of those  
individually through the REST API[2] would appear to be very inefficient.

syslog-nagios-bridge[3] also uses this mechanism to submit results to  
Nagios, but usually not with the same intensity as ganglia-nagios-bridge.

Regards,

Daniel

1. [https://danielpocock.com/ganglia-nagios-bridge](https://danielpocock.com/ganglia-nagios-bridge)  
2. [https://sensuapp.org/docs/latest/api/results-api.html#results-post](https://sensuapp.org/docs/latest/api/results-api.html#results-post)  
3. [https://github.com/dpocock/syslog-nagios-bridge](https://github.com/dpocock/syslog-nagios-bridge)

---

<div class="post-metadata">

**Author:** ![Kyle\_Anderson](https://avatars.discourse-cdn.com/v4/letter/k/e5b9ba/32.png) [@Kyle\_Anderson](https://discourse.sensu.io/u/Kyle_Anderson)\
**Post date:** [October 17, 2016, 2:43pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/2 "2016-10-17T14:43:28Z")

</div>

[https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage](https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage)

Is the socket you want to send bulk check results, but it consumes json of its own specification.

> **···**
>
> On Mon, Oct 17, 2016 at 7:14 AM, Daniel Pocock [daniel@pocock.pro](mailto:daniel@pocock.pro) wrote:
> 
> > Nagios has a spool directory where check results are written into text
> > 
> > files and queued for processing.
> > 
> > It is possible for results from multiple checks to be placed in a single
> > 
> > file, all of those results are then processed more efficiently by the
> > 
> > Nagios code.
> > 
> > ganglia-nagios-bridge[1] extracts all the metrics from a Ganglia gmetad
> > 
> > server in a single poll and dumps them all out to a single Nagios check
> > 
> > results file for bulk processing.
> > 
> > Can Sensu accept these Nagios bulk checkresult files?
> > 
> > If not, is there any alternative way to submit bulk check results from
> > 
> > tools like ganglia-nagios-bridge into Sensu? It has been observed
> > 
> > processing thousands of check results in one go, doing each of those
> > 
> > individually through the REST API[2] would appear to be very inefficient.
> > 
> > syslog-nagios-bridge[3] also uses this mechanism to submit results to
> > 
> > Nagios, but usually not with the same intensity as ganglia-nagios-bridge.
> > 
> > Regards,
> > 
> > Daniel
> > 
> > 1. [https://danielpocock.com/ganglia-nagios-bridge](https://danielpocock.com/ganglia-nagios-bridge)
> > 
> > 2. [https://sensuapp.org/docs/latest/api/results-api.html#results-post](https://sensuapp.org/docs/latest/api/results-api.html#results-post)
> > 
> > 3. [https://github.com/dpocock/syslog-nagios-bridge](https://github.com/dpocock/syslog-nagios-bridge)

---

<div class="post-metadata">

**Author:** ![Daniel\_Pocock](https://avatars.discourse-cdn.com/v4/letter/d/8e8cbc/32.png) [@Daniel\_Pocock](https://discourse.sensu.io/u/Daniel_Pocock)\
**Post date:** [October 17, 2016, 2:46pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/3 "2016-10-17T14:46:03Z")

</div>

Can multiple results be submitted in a single TCP connection?

Or is it necessary to reconnect each time a result is submitted?

Doesn't the client then have to send them over RabbitMQ to the server,  
is there no way to get thousands of check results directly into the  
server process, bypassing the client and RabbitMQ?

> **···**
>
> On 17/10/16 16:43, Kyle Anderson wrote:
> 
> > [https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage](https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage)
> > 
> > Is the socket you want to send bulk check results, but it consumes json  
> > of its own specification.

---

<div class="post-metadata">

**Author:** ![Kyle\_Anderson](https://avatars.discourse-cdn.com/v4/letter/k/e5b9ba/32.png) [@Kyle\_Anderson](https://discourse.sensu.io/u/Kyle_Anderson)\
**Post date:** [October 17, 2016, 2:59pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/4 "2016-10-17T14:59:51Z")

</div>

Hmm. I’ve never tried submitting multiple requests to that socket at a time, but aparently you can, and even keep a consistent connection (but this doesn’t look like it is documented):  
[https://github.com/sensu/sensu/blob/899c298bca5e2c5dfa79a3622cb1b2ee59424f10/lib/sensu/client/socket.rb#L235-L238](https://github.com/sensu/sensu/blob/899c298bca5e2c5dfa79a3622cb1b2ee59424f10/lib/sensu/client/socket.rb#L235-L238)

But even via the API, it is still submitting to rabbitmq for the server to process.

I’m not aware of a method to bypass the transport. Maybe you could write an Extension. (Maybe the server could have a socket?

But if it did, how would you be sure you were talking to the current leader, etc?)

> **···**
>
> On Mon, Oct 17, 2016 at 7:46 AM, Daniel Pocock [daniel@pocock.pro](mailto:daniel@pocock.pro) wrote:
> 
> > On 17/10/16 16:43, Kyle Anderson wrote:
> > 
> > > [https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage](https://sensuapp.org/docs/0.26/reference/clients.html#example-client-socket-usage)
> > 
> > > 
> > 
> > > Is the socket you want to send bulk check results, but it consumes json
> > 
> > > of its own specification.
> > 
> > Can multiple results be submitted in a single TCP connection?
> > 
> > Or is it necessary to reconnect each time a result is submitted?
> > 
> > Doesn’t the client then have to send them over RabbitMQ to the server,
> > 
> > is there no way to get thousands of check results directly into the
> > 
> > server process, bypassing the client and RabbitMQ?

---

<div class="post-metadata">

**Author:** ![Moises\_Silva](https://avatars.discourse-cdn.com/v4/letter/m/e68b1a/32.png) [@Moises\_Silva](https://discourse.sensu.io/u/Moises_Silva)\
**Post date:** [October 17, 2016, 3:51pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/5 "2016-10-17T15:51:51Z")

</div>

Las time I checked, you can't. The tcp socket is read as long as the data  
received fails to parse as a json string or a timeout occurrs (500msec  
iirc). If you try to submit multiple check results, how would you append  
one check result to another? json parsing would fail unless you sent a json  
array of check results, in which case code will fail cuz is not expecting  
an array, but just a json object (hash/dictionary).

> **···**
>
> On Mon, Oct 17, 2016 at 10:59 AM, Kyle Anderson \<kyle@xkyle.com\> wrote:
> 
> > Hmm. I've never tried submitting multiple requests to that socket at a  
> > time, but aparently you can, and even keep a consistent connection (but  
> > this doesn't look like it is documented):  
> > [GitHub - sensu/sensu at 899c298bca5e2c5dfa79a3622cb1b2ee59424f10](https://github.com/sensu/sensu/blob/899c298bca5e2c5dfa79a3622cb1b2)  
> > ee59424f10/lib/sensu/client/socket.rb#L235-L238

---

<div class="post-metadata">

**Author:** ![John\_Fessenden](https://avatars.discourse-cdn.com/v4/letter/j/46a35a/32.png) [@John\_Fessenden](https://discourse.sensu.io/u/John_Fessenden)\
**Post date:** [October 17, 2016, 5:25pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/6 "2016-10-17T17:25:32Z")

</div>

aside: transport (rabbitmq usually) is the servers work queue. you don’t want to bypass it.

> **···**
>
> > On Oct 17, 2016, at 7:46 AM, Daniel Pocock \<daniel@pocock.pro\> wrote:
> > 
> > is there no way to get thousands of check results directly into the  
> > server process, bypassing the client and RabbitMQ?

---

<div class="post-metadata">

**Author:** ![Daniel\_Pocock](https://avatars.discourse-cdn.com/v4/letter/d/8e8cbc/32.png) [@Daniel\_Pocock](https://discourse.sensu.io/u/Daniel_Pocock)\
**Post date:** [October 17, 2016, 5:36pm UTC](https://discourse.sensu.io/t/bulk-check-result-submission-into-sensu/700/7 "2016-10-17T17:36:34Z")

</div>

Would you consider supporting some bulk imports through a dedicated  
queue in either RabbitMQ or directly into Redis perhaps? I notice you  
also support Redis as a transport now, can both Redis and RabbitMQ  
transports be enabled concurrently for different clients?

Something like ganglia-nagios-bridge (or ganglia-sensu-bridge) would  
probably run on the same host as the Sensu server itself.

Regards,

Daniel

> **···**
>
> On 17/10/16 19:25, John Fessenden wrote:
> 
> > aside: transport (rabbitmq usually) is the servers work queue. you don’t want to bypass it.