Trigger events in sensu based on custom severity field ?


#1

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay


#2

Hi Akshay,

Currently, you can use a exit status higher than 3, will be considered “unknown”. Is still available in event data.

Sean.

···

On Apr 27, 2015 12:27 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay


#3

I see that in the code. Now to send probably an alert on status - 5 what part of the config/code would I have to change.

Maybe write some pluggable code which will include this logic or am i getting this wrong.

If yes, could you tell me how to achieve that.

Akshay

···

Sent from my iPhone

On Apr 27, 2015, at 8:05 AM, Sean Porter portertech@sensuapp.com wrote:

Hi Akshay,

Currently, you can use a exit status higher than 3, will be considered “unknown”. Is still available in event data.

Sean.

On Apr 27, 2015 12:27 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay


#4

Your check command can produce an exit status code of 5. You can update your check plugin to do this, how its done depends on the language its written in.

Ruby uses: exit(5)

Sean.

···

On Apr 27, 2015 9:16 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

I see that in the code. Now to send probably an alert on status - 5 what part of the config/code would I have to change.

Maybe write some pluggable code which will include this logic or am i getting this wrong.

If yes, could you tell me how to achieve that.

Akshay

Sent from my iPhone

On Apr 27, 2015, at 8:05 AM, Sean Porter portertech@sensuapp.com wrote:

Hi Akshay,

Currently, you can use a exit status higher than 3, will be considered “unknown”. Is still available in event data.

Sean.

On Apr 27, 2015 12:27 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay


#5

Thanks Sean. I think i’m understanding something incorrectly. I believe sensu-server decides whether the event is to be given to the handler or not based on the value in the status field.

So will i have to make some changes there so as to have an event sent out to the handler when the status is a value of say 5 ? If not can you point me to the code where this is taken care of.

···

On Mon, Apr 27, 2015 at 9:21 AM, Sean Porter portertech@sensuapp.com wrote:

Your check command can produce an exit status code of 5. You can update your check plugin to do this, how its done depends on the language its written in.

Ruby uses: exit(5)

Sean.

On Apr 27, 2015 9:16 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

I see that in the code. Now to send probably an alert on status - 5 what part of the config/code would I have to change.

Maybe write some pluggable code which will include this logic or am i getting this wrong.

If yes, could you tell me how to achieve that.

Akshay

Sent from my iPhone

On Apr 27, 2015, at 8:05 AM, Sean Porter portertech@sensuapp.com wrote:

Hi Akshay,

Currently, you can use a exit status higher than 3, will be considered “unknown”. Is still available in event data.

Sean.

On Apr 27, 2015 12:27 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay


#6

Every event with a non-zero check status will be passed to its event handlers. So an event with a check status of 5 will still be passed to its event handlers.

Sean.

···

On Wed, Apr 29, 2015 at 8:51 AM, Akshay Kapoor akshay.anilkapoor@gmail.com wrote:

Thanks Sean. I think i’m understanding something incorrectly. I believe sensu-server decides whether the event is to be given to the handler or not based on the value in the status field.

So will i have to make some changes there so as to have an event sent out to the handler when the status is a value of say 5 ? If not can you point me to the code where this is taken care of.

On Mon, Apr 27, 2015 at 9:21 AM, Sean Porter portertech@sensuapp.com wrote:

Your check command can produce an exit status code of 5. You can update your check plugin to do this, how its done depends on the language its written in.

Ruby uses: exit(5)

Sean.

On Apr 27, 2015 9:16 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

I see that in the code. Now to send probably an alert on status - 5 what part of the config/code would I have to change.

Maybe write some pluggable code which will include this logic or am i getting this wrong.

If yes, could you tell me how to achieve that.

Akshay

Sent from my iPhone

On Apr 27, 2015, at 8:05 AM, Sean Porter portertech@sensuapp.com wrote:

Hi Akshay,

Currently, you can use a exit status higher than 3, will be considered “unknown”. Is still available in event data.

Sean.

On Apr 27, 2015 12:27 AM, “Akshay Kapoor” akshay.anilkapoor@gmail.com wrote:

Hi,

I want to be able to extend sensu so as to make it capable to understand 6 levels of severity instead of 3 that sensu has (0,1,2). How can i extend so as to support 6 levels. I’m sure that this functionality can be extended in sensu, but not sure how to go about this.

One way i think of doing this is to create a new field in the event data and have my custom severity but when data is sent to sensu it maps it to the 3 severities that sensu understands. Something like this, (Again not sure if i can have all of this logic in the check definition…i guess would be in the check script to add a new field in the event data, like we can add to external check result.)

custom severity - 2,3,4 would map to sensu severity of 1 (WARN)

custom severity - 5,6 would map to sensu severity of 2 (Critical)

The other way would be to trigger events in sensu on a custom field (like a custom field, ‘severity’ for example ) . I looked into the code for sensu but couldn’t find any such functionality. Please correct me if i’m missing something.

If anybody could share some pointers/tips on what is the best way to achieve this in sensu.

Thanks,

Akshay