Introducing curated configurations: a new way to contribute and share operational expertise!

Sensu Go
1

Hey!

As some of you might already be aware, we’ve been working on a new idea to make it possible to share curated Sensu resource configurations and to help bootstrap new Sensu users into effective monitoring workloads more quickly. By making use of Sensu Go features such as assets and secrets management, it’s finally possible to share monitoring workflow configurations widely and safely.

We’ve started the process, created several curated workflows, and we think it’s ready to share, but we need your help to take it further. This is a great opportunity to share operational knowledge with the rest of the Sensu community. If there is a handler or check you have operational experience with, we’d like to invite you to get involved in the configuration curation! If your interested take a look at:

https://github.com/sensu-community/monitoring-checks
https://github.com/sensu-community/monitoring-pipelines

Okay that was the tease, here’s the longer rundown.

We’ve split the configuration curation into two separate repositories. One for checks that run on a Sensu agent, and a separate repository for pipelines that run on a Sensu backend. These repositories are designed to work together, but we felt it made logical sense to split the functionality at the backend/agent boundary.

Take a look at the nginx metrics check from the collection to get a sense of what a curated configuration looks like:

The curated check configurations not only include the check resource but also the versioned asset resources as well. All you need to do is install the single yaml file to have an unpublished check staged into your Sensu namespace that is ready to interact with handler sets defined from the curated monitoring-pipelines. Our hope is these curated checks will get people up and running with common monitoring tasks that can then be easily optimized once installed.

Speaking of, the curated monitoring pipelines on the other hand run on the Sensu backend, and are meant to make it easier for new users to integrate with the service integrations they need to interact with. These curated pipeline configurations make use of Sensu secrets management, so that sensitive information isn’t encoded into the publicly shared resources. Take a look at the curated pagerduty pipeline for example:

This curation defines a handler and an asset, but also defines a Sensu secret resource, to instruct the Sensu backend to pull the pagerduty token from the backend environment. This makes it safe to share the configuration without fear of leaking sensitive information,

Please take a look at these configuration repositories and see if there’s an operational workflow in your environment that you could help curate. We’ve stubbed in the directory structure to help give a sense of the coverage that is possible based on the Sensu plugins that exist, but we could really use operator expertise to fill in the gaps. If you’re interested in helping, take a look at the Readme in the repos, we’ve tried to document what we think makes for a great curated configuration.

We are also planning to hold a workshop session around curations during the upcoming virtual summit, May 19-21. We’d love to work together on this with people who want to contribute to this effort during that workshop. Please register for the our virtual summit and join the live conversation.

1 Like
2

looks like a great start, would be nice to see a reference to were to install the asset from in the check definition itself, so when you create the new check definition, sensu could on its own check for the required assets and install. or if you had an internal asset server, to reference when instead. Also would be nice to have some references for humans, eg were did this check come from? does it have a home page, and is there a playbook to resolving this issue for humans to follow when resolving?

3

Hey!

We’re definitely trying to make sure the yaml files contain useful comment blocks, so its possible for operators to read along through the yaml comments and make appropriate edits to adapt the configuration. I’m not sure i fully understand some of what your suggesting, it might be useful talk through your suggestions for an existing curation, the check-http should be familiar enough for most people reading along.

This yaml file starts with a comment block that includes a reference to the check’s upstream documentation. Are you looking for a more expansive reference as to where the check came from?

I’m not sure what you meant when as “to where to install the asset from the check” in your comments. The yaml defines the check and a both the needed assets together in a single file, pulling assets from the public bonsai asset collection. If a user needs to use a privately hosted asset, it should be a simple matter of editing this file and changing the referenced assets in the check definition. Is there something else you were looking for? Do you have an idea on how to better comment the file?

The playbook comment is a bit broader. We don’t have a space yet for published remediation playbooks that match up with the templates, but if there was interest in operators in our community to share that, we could definitely make a space for that. It’s sort of a chicken and egg problem, I’d love to come back and add curated remediation playbook suggestions into the check comment blocks once we have some curated playbooks published. The related repository for monitoring-pipelines does anticipate the development of curated remediation pipelines. If people wanted to start contributing curated playbooks, they probably could start with PRs to the monitoring-pipelines repository for now.

Thanks for the feedback!
-Jef

4

simply, the check should have the URL to the asset, such that we don’t need to include the required assets in file to get bootstrapped .

5

Ah you want to have a way to encode an equivalent of sensuctl asset add statement inside a check so when creating a check definition and interact with an asset repository api so the asset resource is generated and installed if needed behind the scenes? That’s an interesting idea.

1 Like