Deploying Sensu Plugins are making me gray headed.


#1

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


#2

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

···

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamstracener@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?

Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#3

I do mean use the Ansible gem module and pin the version. The plugins use semantic versioning but I always feel better when my production stuff is pinned.

As far as the embedded Ruby goes, that is up to you. In my environment I use the embedded Ruby for all Sensu stuff. You can use it however you wish and truth be told it may work out for you easier another way depending on your needs but that is more a design question than anything.

Using the embedded ruby with both the client and the server as well as all the plugins is the best way to ensure system performance and stability as the embedded ruby will always be the most heavily targeted from a testing standpoint.

···

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamstracener@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones mattjones@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamstracener@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones

Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#4

Please compare the results of the old sensu plugin system with the new one:

I really like the idea of having each plugin within a seperate gem. But the slow install time is is extremly frustrating.

Do you see any upcoming improvement here? Maybe releasing these plugins as a normal deb/rpm packages alongside sensu?

···

Am Dienstag, 28. Juli 2015 04:21:09 UTC+2 schrieb Matt Jones:

I do mean use the Ansible gem module and pin the version. The plugins use semantic versioning but I always feel better when my production stuff is pinned.

As far as the embedded Ruby goes, that is up to you. In my environment I use the embedded Ruby for all Sensu stuff. You can use it however you wish and truth be told it may work out for you easier another way depending on your needs but that is more a design question than anything.

Using the embedded ruby with both the client and the server as well as all the plugins is the best way to ensure system performance and stability as the embedded ruby will always be the most heavily targeted from a testing standpoint.

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamst...@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones matt...@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamst...@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#5

The reason for the slow install is generally due to the dependencies. This is ongoing and the more we as a community can refactor out some of the uglier ones the better.

Rpm’s are not completely out of the question but that would be a huge undertaking and would defeat the purpose and some of the design of sensu.

The community is welcome to discuss this and any other option and no good ideas will be dismissed but we are a small team with limited resources.

···

On Jul 28, 2015 7:44 AM, “Philipp H” hellmi@gmail.com wrote:

Please compare the results of the old sensu plugin system with the new one:
https://github.com/sensu-plugins/sensu-plugin/issues/93

I really like the idea of having each plugin within a seperate gem. But the slow install time is is extremly frustrating.

Do you see any upcoming improvement here? Maybe releasing these plugins as a normal deb/rpm packages alongside sensu?

Am Dienstag, 28. Juli 2015 04:21:09 UTC+2 schrieb Matt Jones:

I do mean use the Ansible gem module and pin the version. The plugins use semantic versioning but I always feel better when my production stuff is pinned.

As far as the embedded Ruby goes, that is up to you. In my environment I use the embedded Ruby for all Sensu stuff. You can use it however you wish and truth be told it may work out for you easier another way depending on your needs but that is more a design question than anything.

Using the embedded ruby with both the client and the server as well as all the plugins is the best way to ensure system performance and stability as the embedded ruby will always be the most heavily targeted from a testing standpoint.

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamst...@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones matt...@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamst...@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#6

I think there are two problems:

  1. Slow install time of any plugin due to having to fetch the gemspec and resolve all dependecies (we could tune some plugins here, but in general we also depdend on other plugins which are responsible for a big dependency tree)
  2. chef sensu_gem executes “gem pluginname” for each plugin, this means the latest_specs file gets downladed multiple times (ansible and puppet might have the some problem)

If you run a gem install sensu-plugin-http sensu-plugin-nginx, the gemspec is only loaded once.

Additionally a local gem cache would reduce the download time. Maybe even the newest rubygems package might improve the install speed.

Sensu itself is also delivered as a system package and nagios-plugins is also a system package. Which design rules aren’t met for a sensu-plugins(-category?) package here?

···

Am Dienstag, 28. Juli 2015 13:54:56 UTC+2 schrieb Matt Jones:

The reason for the slow install is generally due to the dependencies. This is ongoing and the more we as a community can refactor out some of the uglier ones the better.

Rpm’s are not completely out of the question but that would be a huge undertaking and would defeat the purpose and some of the design of sensu.

The community is welcome to discuss this and any other option and no good ideas will be dismissed but we are a small team with limited resources.

On Jul 28, 2015 7:44 AM, “Philipp H” hel...@gmail.com wrote:

Please compare the results of the old sensu plugin system with the new one:
https://github.com/sensu-plugins/sensu-plugin/issues/93

I really like the idea of having each plugin within a seperate gem. But the slow install time is is extremly frustrating.

Do you see any upcoming improvement here? Maybe releasing these plugins as a normal deb/rpm packages alongside sensu?

Am Dienstag, 28. Juli 2015 04:21:09 UTC+2 schrieb Matt Jones:

I do mean use the Ansible gem module and pin the version. The plugins use semantic versioning but I always feel better when my production stuff is pinned.

As far as the embedded Ruby goes, that is up to you. In my environment I use the embedded Ruby for all Sensu stuff. You can use it however you wish and truth be told it may work out for you easier another way depending on your needs but that is more a design question than anything.

Using the embedded ruby with both the client and the server as well as all the plugins is the best way to ensure system performance and stability as the embedded ruby will always be the most heavily targeted from a testing standpoint.

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamst...@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones matt...@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamst...@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#7

So Sensu is ran under the Sensu user which does not allow login. So how would that work with Ansible?

If I define the executable as /opt/sensu/embedded/bin/gem in the play how would I say to do the gem install as Sensu and not my deployment account?

I’ve also noticed that even if I set EMBEDDED_RUBY=true the Path is not being set to use the embedded ruby it’s still pointing to /var/lib/ruby1.9.1.

Should no other ruby package be installed if I plan on using embedded ruby?

···

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamstracener@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones mattjones@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamstracener@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones

Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones


#8

I could be mistaken but you have a few options here.

log in and then do an su - sensu or I believe there is a way in Ansible to run command as. Worse case just do a chown after the fact.

You can install what you want, every gem I install uses the embedded ruby. It’s there and I have no special uses cases for other versions so rather than try and maintain my own I just use it.

···

On Tue, Jul 28, 2015 at 8:52 PM, Adam Stracener adamstracener@gmail.com wrote:

So Sensu is ran under the Sensu user which does not allow login. So how would that work with Ansible?

If I define the executable as /opt/sensu/embedded/bin/gem in the play how would I say to do the gem install as Sensu and not my deployment account?

I’ve also noticed that even if I set EMBEDDED_RUBY=true the Path is not being set to use the embedded ruby it’s still pointing to /var/lib/ruby1.9.1.

Should no other ruby package be installed if I plan on using embedded ruby?

Sent from my iPhone

On Jul 27, 2015, at 9:21 PM, Matt Jones mattjones@yieldbot.com wrote:

I do mean use the Ansible gem module and pin the version. The plugins use semantic versioning but I always feel better when my production stuff is pinned.

As far as the embedded Ruby goes, that is up to you. In my environment I use the embedded Ruby for all Sensu stuff. You can use it however you wish and truth be told it may work out for you easier another way depending on your needs but that is more a design question than anything.

Using the embedded ruby with both the client and the server as well as all the plugins is the best way to ensure system performance and stability as the embedded ruby will always be the most heavily targeted from a testing standpoint.

On Mon, Jul 27, 2015 at 9:52 PM, Adam Stracener adamstracener@gmail.com wrote:

Why embedded ruby over system? Should just the clients use embedded or the server as well?

When you say pin the version I assume you mean use Ansibles gem module and specify the version?

Sent from my iPhone

On Jul 27, 2015, at 8:28 PM, Matt Jones mattjones@yieldbot.com wrote:

I am using Chef at the moment but have used Ansible in the past for other purposes and the principles hold the same.

I would set up some sort of playbook for just the plugins and pin each gem to the version you want. You could use bundler or other methods but this seems the easiest in my mind. From there you can just include that playbook on each machine that has a need for the plugins. This does mean that each machine would have all the plugin used in the environment but you can split things how you want if you have a need for more control.

Concerning the user and ruby version, I would just run the playbook as the same user that the client runs as and install them using the embedded ruby. You could use other methods but this is by far the cleanest method and is how I do it with Chef.


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones

On Mon, Jul 27, 2015 at 8:56 PM, Adam Stracener adamstracener@gmail.com wrote:

Up until the past two or three weeks we have been using the old way of handling the Sensu Plugins. Since they have now moved them into their own repo I’m having a hard time understand the best way to deploy the plugins between all of my clients. What is the best method? I use Ansible for deployment, should I just install each gem file individually? Should I use a Gemfile and Bundler? I want to make sure that all the plugins are kept up to date on each client. What’s the best way to handle this?

Which user should install the gem file? Sensu? My maintenance account? Root?


Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones

Matt Jones @DevopsMatt

Infrastructure Engineer - Yieldbot Inc.

Core Contributor - Sensu Plugins

https://linkedin.com/in/mattyjones