Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
I’d through in statsd also here. I’m in the process of looking at each also.
It seems ATM, statsd = nodejs, collectd = compiled c prog, and senus = bash?
Other than that; not too sure what else…
···
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:06 PM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
For us, the decision was made to have a single message bus for events (e.g. alerts, metrics). Sensu has served this purpose well for us.
All of our metrics are collected via Ruby or Java applications. Sensu can effectively run any application to generate statistics. It also has the ability to have metrics pushed to it, so instrumented code can be made to push metrics and alerts through Sensu (we are releasing both a Java and Ruby API that application developers can use to do this sometime in the not too distant future).
···
On Tue, Dec 17, 2013 at 10:22 AM, JJ Asghar jjasghar@gmail.com wrote:
I’d through in statsd also here. I’m in the process of looking at each also.
It seems ATM, statsd = nodejs, collectd = compiled c prog, and senus = bash?
Other than that; not too sure what else…
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:06 PM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
···
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more 
···
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
···
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
···
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjasghar@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
···
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller joeym@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjasghar@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
···
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agent462@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller joeym@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjasghar@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
···
2013/12/17 Joe Miller joeym@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agent462@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller joeym@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjasghar@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Thanks,
It all really comes down to the use case and scale. In some situations it would be really nice to just pipeline everything through sensu. The flexibility of sensu is amazing and we all love ruby…seriously who doesn’t love ruby. At the same time you need to take your scale into consideration.
We try to stick to our outlined ways of collecting our metrics. That way in our larger team your not digging through a repo or shouting to everyone how the metric is collected. I’d pick your standard and do as much as you can with that.
In our setup I can worry about scaling each piece of our monitoring tier individually. We didn’t even have to change our sensu setup for our holiday scaling this year. We did have to scale out our graphite with more relay and cache processes though. My point, I didn’t have to worry about trying to scale sensu and graphite because they were bolted together. It also makes it easier to swap out tools faster when they’re not all bolted together.
Anyway, that’s my .02.
-Bryan
···
On Tue, Dec 17, 2013 at 3:56 PM, Alejandro cdgraff@gmail.com wrote:
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
2013/12/17 Joe Miller joeym@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agent462@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller joeym@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjasghar@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjasghar@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdgraff@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller joeym@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Hi Bryan.
Just read the thread and I’m curious about your implementation: “We have sensu checks that get json data from graphite to inspect the values for alerting”. Let me ask you how you guys do this? I mean, is that a simple Graphite API request that will ask for X metric value and then evaluate it?
Thanks!
···
El martes, 17 de diciembre de 2013 19:07:31 UTC-3, agent462 escribió:
Thanks,
It all really comes down to the use case and scale. In some situations it would be really nice to just pipeline everything through sensu. The flexibility of sensu is amazing and we all love ruby…seriously who doesn’t love ruby. At the same time you need to take your scale into consideration.
We try to stick to our outlined ways of collecting our metrics. That way in our larger team your not digging through a repo or shouting to everyone how the metric is collected. I’d pick your standard and do as much as you can with that.
In our setup I can worry about scaling each piece of our monitoring tier individually. We didn’t even have to change our sensu setup for our holiday scaling this year. We did have to scale out our graphite with more relay and cache processes though. My point, I didn’t have to worry about trying to scale sensu and graphite because they were bolted together. It also makes it easier to swap out tools faster when they’re not all bolted together.
Anyway, that’s my .02.
-Bryan
On Tue, Dec 17, 2013 at 3:56 PM, Alejandro cdg...@gmail.com wrote:
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
2013/12/17 Joe Miller jo...@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agen...@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller jo...@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjas...@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjas...@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdg...@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller jo...@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdg...@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Yes! We run a modified version of the following plugin: https://github.com/sensu/sensu-community-plugins/blob/master/plugins/graphite/check-data.rb
The Graphite render api can return json by passing in format=json. You then just inspect those values.
Thanks,
Bryan
···
On Mon, Feb 17, 2014 at 10:42 AM, Mariano González gonzalez.mariano.gabriel@gmail.com wrote:
Hi Bryan.
Just read the thread and I’m curious about your implementation: “We have sensu checks that get json data from graphite to inspect the values for alerting”. Let me ask you how you guys do this? I mean, is that a simple Graphite API request that will ask for X metric value and then evaluate it?
Thanks!
El martes, 17 de diciembre de 2013 19:07:31 UTC-3, agent462 escribió:
Thanks,
It all really comes down to the use case and scale. In some situations it would be really nice to just pipeline everything through sensu. The flexibility of sensu is amazing and we all love ruby…seriously who doesn’t love ruby. At the same time you need to take your scale into consideration.
We try to stick to our outlined ways of collecting our metrics. That way in our larger team your not digging through a repo or shouting to everyone how the metric is collected. I’d pick your standard and do as much as you can with that.
In our setup I can worry about scaling each piece of our monitoring tier individually. We didn’t even have to change our sensu setup for our holiday scaling this year. We did have to scale out our graphite with more relay and cache processes though. My point, I didn’t have to worry about trying to scale sensu and graphite because they were bolted together. It also makes it easier to swap out tools faster when they’re not all bolted together.
Anyway, that’s my .02.
-Bryan
On Tue, Dec 17, 2013 at 3:56 PM, Alejandro cdg...@gmail.com wrote:
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
2013/12/17 Joe Miller jo...@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agen...@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller jo...@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjas...@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjas...@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdg...@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller jo...@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdg...@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Hi Bryan,
I’ll like to know if still be the same stack mention a few years ago, or has changed in some way…
Thanks,
Alejandro
···
El lunes, 17 de febrero de 2014, 13:54:43 (UTC-3), agent462 escribió:
Yes! We run a modified version of the following plugin: https://github.com/sensu/sensu-community-plugins/blob/master/plugins/graphite/check-data.rb
The Graphite render api can return json by passing in
format=json. You then just inspect those values.
Thanks,
Bryan
On Mon, Feb 17, 2014 at 10:42 AM, Mariano González gonzalez.mar...@gmail.com wrote:
Hi Bryan.
Just read the thread and I’m curious about your implementation: “We have sensu checks that get json data from graphite to inspect the values for alerting”. Let me ask you how you guys do this? I mean, is that a simple Graphite API request that will ask for X metric value and then evaluate it?
Thanks!
El martes, 17 de diciembre de 2013 19:07:31 UTC-3, agent462 escribió:
Thanks,
It all really comes down to the use case and scale. In some situations it would be really nice to just pipeline everything through sensu. The flexibility of sensu is amazing and we all love ruby…seriously who doesn’t love ruby. At the same time you need to take your scale into consideration.
We try to stick to our outlined ways of collecting our metrics. That way in our larger team your not digging through a repo or shouting to everyone how the metric is collected. I’d pick your standard and do as much as you can with that.
In our setup I can worry about scaling each piece of our monitoring tier individually. We didn’t even have to change our sensu setup for our holiday scaling this year. We did have to scale out our graphite with more relay and cache processes though. My point, I didn’t have to worry about trying to scale sensu and graphite because they were bolted together. It also makes it easier to swap out tools faster when they’re not all bolted together.
Anyway, that’s my .02.
-Bryan
On Tue, Dec 17, 2013 at 3:56 PM, Alejandro cdg...@gmail.com wrote:
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
2013/12/17 Joe Miller jo...@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agen...@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller jo...@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjas...@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjas...@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdg...@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller jo...@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdg...@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro
Alejandro,
From this aspesct it’s still pretty much the same. It works very well for us and has been a solid setup.
Thanks,
Bryan
···
On Mon, Jul 27, 2015 at 8:46 PM, Alejandro Ferrari cdgraff@gmail.com wrote:
Hi Bryan,
I’ll like to know if still be the same stack mention a few years ago, or has changed in some way…
Thanks,
Alejandro
El lunes, 17 de febrero de 2014, 13:54:43 (UTC-3), agent462 escribió:
Yes! We run a modified version of the following plugin: https://github.com/sensu/sensu-community-plugins/blob/master/plugins/graphite/check-data.rb
The Graphite render api can return json by passing in
format=json. You then just inspect those values.
Thanks,
Bryan
On Mon, Feb 17, 2014 at 10:42 AM, Mariano González gonzalez.mar...@gmail.com wrote:
Hi Bryan.
Just read the thread and I’m curious about your implementation: “We have sensu checks that get json data from graphite to inspect the values for alerting”. Let me ask you how you guys do this? I mean, is that a simple Graphite API request that will ask for X metric value and then evaluate it?
Thanks!
El martes, 17 de diciembre de 2013 19:07:31 UTC-3, agent462 escribió:
Thanks,
It all really comes down to the use case and scale. In some situations it would be really nice to just pipeline everything through sensu. The flexibility of sensu is amazing and we all love ruby…seriously who doesn’t love ruby. At the same time you need to take your scale into consideration.
We try to stick to our outlined ways of collecting our metrics. That way in our larger team your not digging through a repo or shouting to everyone how the metric is collected. I’d pick your standard and do as much as you can with that.
In our setup I can worry about scaling each piece of our monitoring tier individually. We didn’t even have to change our sensu setup for our holiday scaling this year. We did have to scale out our graphite with more relay and cache processes though. My point, I didn’t have to worry about trying to scale sensu and graphite because they were bolted together. It also makes it easier to swap out tools faster when they’re not all bolted together.
Anyway, that’s my .02.
-Bryan
On Tue, Dec 17, 2013 at 3:56 PM, Alejandro cdg...@gmail.com wrote:
wow, this thread be really useful, thanks for all the comments!
@Bryan, I really like your setup, is simple and have all the features.
@Joe, your point is good to have on mind, possible useful in case of custom metrics that only need into some servers or for limited time… Sensu allow to implement this very quick and fast way.
2013/12/17 Joe Miller jo...@joeym.net
Good point. Sensu wasn’t necessarily meant to be a highly performant metrics bus. That said, there are some metrics that can be easier to collect with sensu when you need to write a quick script.
If you artificially limit yourself to only 1 method for shuttling metrics around then think about your scale.
On Tue, Dec 17, 2013 at 1:40 PM, Bryan Brandau agen...@gmail.com wrote:
We have collectd, sensu and graphite in our environment. We go directly from collectd to graphite. We have sensu checks that get json data from graphite to inspect the values for alerting. Collectd is fast, offers buffering and persistent connections and works well at our scale. We also use logster to collect other metrics from our log files.
Sending every metric through Sensu would have introduced a whole different scale pattern we didn’t want to have with Sensu.
-Bryan
On Tue, Dec 17, 2013 at 2:33 PM, Joe Miller jo...@joeym.net wrote:
There are statsd server implementations in almost every language. I try to maintain a list here: http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/
FWIW, we are a mostly a python shop with some nodejs and ruby shop. We started with one of the python statsd implementations but it was buggy and the maintainer was unresponsive. We switched to the “official” node.js statsd a few months ago because it was just much better in terms of bug fixes, features, etc. Getting it running and configuring it is pretty easy and doesn’t require too much are and feeding.
On Tue, Dec 17, 2013 at 11:41 AM, JJ Asghar jjas...@gmail.com wrote:
For the lazy:
http://joemiller.me/2011/04/14/collectd-graphite-plugin/
I’m really interested in statsd+sensu+graphite. Granted I’m at a rails shop so adding nodejs everywhere seems…more over head that it’s worth. Any suggestions on this?
Right now for trending i’m using ganglia+gmond, and i want to move away from it.
Any suggestions?
Best Regards,
JJ Asghar | jjas...@gmail.com | p: 512.619.0722
On Tue, Dec 17, 2013 at 12:52 PM, Alejandro cdg...@gmail.com wrote:
My question come after test many stacks, right now we use Logstash and Sensu, now Logstash have CollectD input too, and googling I found your blog (Joe) where you talk about collectd and graphite.
I’m trying to have the most complete but simple stack possible, no 2 script for the same issue, have metrics and check in 2 scripts I think is not the best, how say before, is just research about the best way to connect all the tools available without have duplication.
Thanks for all the comments, hope read more
2013/12/17 Joe Miller jo...@joeym.net
At the first place I deployed sensu I used both Sensu + Collectd. Collectd was mostly used for the basic plugins that it ships with: disk, cpu, net, memory. Sensu was later added to the environment for monitoring/alerting and because it’s easy to write quick scripts to gather metrics we started writing sensu scripts when we needed to collect some new metric, typically an application metric or a metric that didn’t have a relevant collectd plugin already.
In this environment sensu and collectd → graphite worked well.
In my current environment, the situation is similar. We have munin feeding some metrics to graphite and sensu feeding some. Munin may eventually go away but it works just fine so there is no rush.
In both cases the decision was made to collect metrics with the least friction rather than picking only one approach. The pattern has worked out fine for us.
On Tue, Dec 17, 2013 at 10:06 AM, Alejandro Ferrari cdg...@gmail.com wrote:
Hi guys,
What is your opinion about use Sensu metrics or Collectd?
In both cases the backend be Graphite, but my idea is not have duplicated script for the same things.
Any advice be welcome!
Thanks
Alejandro