# How can I aggregate check results to avoid event floods?

**URL:** <https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21>\
**Category:** Sensu Classic (EOL)\
**Created:** [May 8, 2013, 6:38pm UTC](https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21 "2013-05-08T18:38:40Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joe\_Miller](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/joe_miller/32/160_2.png) [@Joe\_Miller](https://discourse.sensu.io/u/Joe_Miller)\
**Post date:** [May 8, 2013, 6:38pm UTC](https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21/1 "2013-05-08T18:38:40Z")

</div>

Sensu supports ‘aggregate’ checks. They are not well documented at this time.

The basics of it are:

1. set 'aggregate:true and ‘handle’:false on the check definition. This prevents the server from sending these to a handler, and makes the aggregated results available under /aggregates in the sensu REST API.

2. setup a check with something like check-aggregate.rb (from the sensu-community-plugins repo) that queries the /aggregates API and takes action on the condition you specify (10%, 50%, etc)

I have not actually setup an aggregate check yet. So if anyone who has can correct any errors I made, please do 😉

> **···**
>
> On Wed, May 8, 2013 at 8:08 AM, Adam Steffes [adamsteffes@gmail.com](mailto:adamsteffes@gmail.com) wrote:
> 
> > Hi, folks. I’d like to have some logic analogous to a Nagios servicegroup or hostgroup such that Sensu will fire a handler only when, say, 5 out of 10 checks have failed. How can I do this?

---

<div class="post-metadata">

**Author:** ![John\_Dyer](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/john_dyer/32/79_2.png) [@John\_Dyer](https://discourse.sensu.io/u/John_Dyer)\
**Post date:** [May 25, 2014, 8:14pm UTC](https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21/2 "2014-05-25T20:14:21Z")

</div>

So in my scenario I have certain checks which I always expect to fail. One example of this is my monitoring for ActiveMQ, which we have deployed as a master/slave pair. In this deployment topology the slave is never really started until it gets the lock on the table once the master dies. This being the case it is no surprise my checks for queue depth via the rest API on that slave fail, since its not actually fully operational when they are run. So in my environment we used the aggregate approach to solve the handler / notification problem which worked great as Joe suggested. However there seems to be one drawback… The dashboard seems to be noisy with these checks that don’t actually require handling by anyone. IMHO the dashboard events page should only contain events which require action by a human being, so this extra noise on the board is a bit of an annoyance. Does anyone know of a way to exclude these from the events pane of Sensu-Admin so my team only see’s alerts which actually require their attention ?

- John

> **···**
>
> On Wednesday, May 8, 2013 2:38:40 PM UTC-4, Joe Miller wrote:
> 
> > Sensu supports ‘aggregate’ checks. They are not well documented at this time.
> 
> > The basics of it are:
> 
> > 1. set 'aggregate:true and ‘handle’:false on the check definition. This prevents the server from sending these to a handler, and makes the aggregated results available under /aggregates in the sensu REST API.
> 
> > 1. setup a check with something like check-aggregate.rb (from the sensu-community-plugins repo) that queries the /aggregates API and takes action on the condition you specify (10%, 50%, etc)
> 
> > 
> 
> > I have not actually setup an aggregate check yet. So if anyone who has can correct any errors I made, please do 😉
> 
> > 
> 
> > On Wed, May 8, 2013 at 8:08 AM, Adam Steffes [adams...@gmail.com](mailto:adams...@gmail.com) wrote:
> > 
> > > Hi, folks. I’d like to have some logic analogous to a Nagios servicegroup or hostgroup such that Sensu will fire a handler only when, say, 5 out of 10 checks have failed. How can I do this?
> 
> >

---

<div class="post-metadata">

**Author:** ![Kyle\_Anderson](https://avatars.discourse-cdn.com/v4/letter/k/e5b9ba/32.png) [@Kyle\_Anderson](https://discourse.sensu.io/u/Kyle_Anderson)\
**Post date:** [May 26, 2014, 3:13am UTC](https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21/3 "2014-05-26T03:13:57Z")

</div>

You could do a check dependency, and have a check that is like  
"check\_activemq\_lock", and have your other checks be dependent on it.  
The lock check would be "red" if it doesn't have a lock, and that  
would suppress handlers from acting on your other activemq checks  
unless the server had the master lock.

But that doesn't solve your problem of dashboard noise.

The stock sensu dashboard is very simple, it basically just reveals  
what the API has. The only way make these not show up in the API is to  
make them actually not fail. This would require modifying the check to  
be something like:

if has\_master\_lock  
&nbsp;&nbsp;check\_activemq\_depth  
else  
&nbsp;&nbsp;puts "OK: Not checking queue depth on a slave"  
&nbsp;&nbsp;exit 0  
end

I know, not super ☹

To me, the complexity of aggregates doesn't really fit this particular  
situation, but the above wrapper is the best I can think of to meet  
your needs, as it actually reflects your alert logic. (The return code  
of the queue depth check depends on the lock state)

> **···**
>
> On Sun, May 25, 2014 at 1:14 PM, John Dyer \<johntdyer@gmail.com\> wrote:
> 
> > &nbsp;&nbsp;&nbsp;&nbsp;So in my scenario I have certain checks which I always expect to fail.  
> > One example of this is my monitoring for ActiveMQ, which we have deployed as  
> > a master/slave pair. In this deployment topology the slave is never really  
> > started until it gets the lock on the table once the master dies. This  
> > being the case it is no surprise my checks for queue depth via the rest API  
> > on that slave fail, since its not actually fully operational when they are  
> > run. So in my environment we used the aggregate approach to solve the  
> > handler / notification problem which worked great as Joe suggested. However  
> > there seems to be one drawback... The dashboard seems to be noisy with these  
> > checks that don't actually require handling by anyone. IMHO the dashboard  
> > events page should only contain events which require action by a human  
> > being, so this extra noise on the board is a bit of an annoyance. Does  
> > anyone know of a way to exclude these from the events pane of Sensu-Admin so  
> > my team only see's alerts which actually require their attention ?
> > 
> > - John
> > 
> > On Wednesday, May 8, 2013 2:38:40 PM UTC-4, Joe Miller wrote:
> > 
> > > Sensu supports 'aggregate' checks. They are not well documented at this  
> > > time.
> > > 
> > > The basics of it are:  
> > > 1) set 'aggregate:true and 'handle':false on the check definition. This  
> > > prevents the server from sending these to a handler, and makes the  
> > > aggregated results available under /aggregates in the sensu REST API.  
> > > 2) setup a check with something like check-aggregate.rb (from the  
> > > sensu-community-plugins repo) that queries the /aggregates API and takes  
> > > action on the condition you specify (10%, 50%, etc)
> > > 
> > > I have not actually setup an aggregate check yet. So if anyone who has can  
> > > correct any errors I made, please do 😉
> > > 
> > > On Wed, May 8, 2013 at 8:08 AM, Adam Steffes \<adams...@gmail.com\> wrote:
> > > 
> > > > Hi, folks. I'd like to have some logic analogous to a Nagios servicegroup  
> > > > or hostgroup such that Sensu will fire a handler only when, say, 5 out of 10  
> > > > checks have failed. How can I do this?

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [November 22, 2018, 2:04am UTC](https://discourse.sensu.io/t/how-can-i-aggregate-check-results-to-avoid-event-floods/21/4 "2018-11-22T02:04:51Z")

</div>