# System to monitor sensu!?

**URL:** <https://discourse.sensu.io/t/system-to-monitor-sensu/122>\
**Category:** Sensu Classic (EOL)\
**Created:** [January 15, 2014, 1:34am UTC](https://discourse.sensu.io/t/system-to-monitor-sensu/122 "2014-01-15T01:34:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Micah\_Hoffmann](https://avatars.discourse-cdn.com/v4/letter/m/7ba0ec/32.png) [@Micah\_Hoffmann](https://discourse.sensu.io/u/Micah_Hoffmann)\
**Post date:** [January 15, 2014, 1:34am UTC](https://discourse.sensu.io/t/system-to-monitor-sensu/122/1 "2014-01-15T01:34:58Z")

</div>

Hey everyone, yesterday our sensu stopped emailing us and it took a little bit for me to notice. I would like to be a little more proactive and put in place some monitoring to monitor our monitoring 🙂

Do you all have any ideas or systems you use?

Thanks

---

<div class="post-metadata">

**Author:** ![Kyle\_Anderson](https://avatars.discourse-cdn.com/v4/letter/k/e5b9ba/32.png) [@Kyle\_Anderson](https://discourse.sensu.io/u/Kyle_Anderson)\
**Post date:** [January 15, 2014, 4:37pm UTC](https://discourse.sensu.io/t/system-to-monitor-sensu/122/2 "2014-01-15T16:37:57Z")

</div>

I'm also in the need of such a thing as I test my sensu clusters to  
replace our nagios clusters.

I was thinking of writing a kind of "end-to-end" sensu check that our  
nagios servers could nrpe.

I'm thinking it would do things in order, and be smart about the  
output to help other members of my ops team diagnose what might be  
wrong: (as sensu has lots of pieces)  
- check ping? Is the sensu vip even pinging?  
- check redis  
- check rabbitmq port  
- check rabbitmq end-to-end (publish and consume a test message?)  
- check sensu-server? (make sure the server is at least running)  
- check handler\_\* (try to make a handler do something and make sure it  
did something?)  
- check dashboard (probably just warn if down for me)

Yes, you could go through the work of setting up individual checks and  
dependencies and stuff in nagios, but I want to avoid that as I am  
trying to deprecate it.

At least I can start easy and get progressively more complex as I  
iterate. I should be able to leverage existing nagios check scripts  
that I have to do all of this easily. (except for handler checks?  
Might be harder, but would be nice to know if sensu can't send emails  
or whatever)

So those are my ideas. In my case my system is nagios, but it could be  
as simple as an xinetd that returns http 200 or http 500 to like a  
pingdom http check or whatever?

I haven't written this yet, but I need to soon. I am also up for  
suggestions and input from other sensu-users.

> **···**
>
> On Tue, Jan 14, 2014 at 5:34 PM, Micah Hoffmann \<micah@pointinside.com\> wrote:
> 
> > Hey everyone, yesterday our sensu stopped emailing us and it took a little  
> > bit for me to notice. I would like to be a little more proactive and put in  
> > place some monitoring to monitor our monitoring 🙂
> > 
> > Do you all have any ideas or systems you use?
> > 
> > Thanks

---

<div class="post-metadata">

**Author:** ![Joe\_Miller](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/joe_miller/32/160_2.png) [@Joe\_Miller](https://discourse.sensu.io/u/Joe_Miller)\
**Post date:** [January 15, 2014, 5:06pm UTC](https://discourse.sensu.io/t/system-to-monitor-sensu/122/3 "2014-01-15T17:06:52Z")

</div>

The sensu-api exposes a /health endpoint since 0.9.13 - [http://sensuapp.org/docs/0.12/api-health](http://sensuapp.org/docs/0.12/api-health)

It will help you determine if there are consumers (sensu-server instances) connected to rabbit and how many msgs are in the results queue.

That said, I’m a pessimist sysadmin and I have never been 100% comfortable with /health as the only way to check sensu’s health. Something that did a more thorough end-to-end automated check all the way through to a handler would be interesting, please share what you come up with!

Occasionally, on an ad-hoc basis, we will send an event via netcat and make sure that we can see it on both our sensu-dashboard and that it makes it to pagerduty, eg: [https://gist.github.com/nstielau/3797054](https://gist.github.com/nstielau/3797054) (change the handler to your pagerduty handler of course)

> **···**
>
> On Wed, Jan 15, 2014 at 8:37 AM, Kyle Anderson [kyle@xkyle.com](mailto:kyle@xkyle.com) wrote:
> 
> > I’m also in the need of such a thing as I test my sensu clusters to
> > 
> > replace our nagios clusters.
> > 
> > I was thinking of writing a kind of “end-to-end” sensu check that our
> > 
> > nagios servers could nrpe.
> > 
> > I’m thinking it would do things in order, and be smart about the
> > 
> > output to help other members of my ops team diagnose what might be
> > 
> > wrong: (as sensu has lots of pieces)
> > 
> > - check ping? Is the sensu vip even pinging?
> > 
> > - check redis
> > 
> > - check rabbitmq port
> > 
> > - check rabbitmq end-to-end (publish and consume a test message?)
> > 
> > - check sensu-server? (make sure the server is at least running)
> > 
> > - check handler\_\* (try to make a handler do something and make sure it
> > 
> > did something?)
> > 
> > - check dashboard (probably just warn if down for me)
> > 
> > Yes, you could go through the work of setting up individual checks and
> > 
> > dependencies and stuff in nagios, but I want to avoid that as I am
> > 
> > trying to deprecate it.
> > 
> > At least I can start easy and get progressively more complex as I
> > 
> > iterate. I should be able to leverage existing nagios check scripts
> > 
> > that I have to do all of this easily. (except for handler checks?
> > 
> > Might be harder, but would be nice to know if sensu can’t send emails
> > 
> > or whatever)
> > 
> > So those are my ideas. In my case my system is nagios, but it could be
> > 
> > as simple as an xinetd that returns http 200 or http 500 to like a
> > 
> > pingdom http check or whatever?
> > 
> > I haven’t written this yet, but I need to soon. I am also up for
> > 
> > suggestions and input from other sensu-users.
> > 
> > On Tue, Jan 14, 2014 at 5:34 PM, Micah Hoffmann [micah@pointinside.com](mailto:micah@pointinside.com) wrote:
> > 
> > > Hey everyone, yesterday our sensu stopped emailing us and it took a little
> > 
> > > bit for me to notice. I would like to be a little more proactive and put in
> > 
> > > place some monitoring to monitor our monitoring 🙂
> > 
> > > 
> > 
> > > Do you all have any ideas or systems you use?
> > 
> > > 
> > 
> > > Thanks

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [November 22, 2018, 2:05am UTC](https://discourse.sensu.io/t/system-to-monitor-sensu/122/4 "2018-11-22T02:05:05Z")

</div>