How to make check for check-log.rb

Romario_Lodder · July 3, 2015, 8:47pm

Hi,

I'm using the check_log to check my logfiles on certain keywords, for example "error". If a new error is written to the log file, sensu gives me an error. After a couple of seconds, the critical error disappears. How should I configure my check in order to check the log file on a word like ERROR and that the error stays as long the error exists in the log file?

My check looks a bit like this:

{
"checks": {
"check_log": {
"command": "/etc/sensu/plugins/check-log.rb -f /opt/test/logs/testDocker.log -q 'ERROR'",
"interval": 10,
"subscribers": ["subscriber1"]
}
}
}

Thanks in advance!

Kyle_Anderson · July 5, 2015, 12:31am

The community check-log is not designed to work like that:

github.com

sensu/sensu-community-plugins/blob/af719c4a11a088ad5f601520a95ff1dd09ea31d9/plugins/logging/check-log.rb#L6


      
          #! /usr/bin/env ruby
          #
          #   check-log
          #
          # DESCRIPTION:
          #   This plugin checks a log file for a regular expression, skipping lines
          #   that have already been read, like Nagios's check_log. However, instead
          #   of making a backup copy of the whole log file (very slow with large
          #   logs), it stores the number of bytes read, and seeks to that position
          #   next time.
          #
          # OUTPUT:
          #   plain text
          #
          # PLATFORMS:
          #   Linux

You could use any nagios-compliant check-grep style script, or even
KISS with "! grep ERROR /opt/test/logs/testDocker.log"? (only would
ever return ok or warn though)

···

On Fri, Jul 3, 2015 at 1:47 PM, Romario Lodder <romariolodder@gmail.com> wrote:

Hi,

I'm using the check_log to check my logfiles on certain keywords, for example "error". If a new error is written to the log file, sensu gives me an error. After a couple of seconds, the critical error disappears. How should I configure my check in order to check the log file on a word like ERROR and that the error stays as long the error exists in the log file?

My check looks a bit like this:

{
"checks": {
"check_log": {
"command": "/etc/sensu/plugins/check-log.rb -f /opt/test/logs/testDocker.log -q 'ERROR'",
"interval": 10,
"subscribers": ["subscriber1"]
}
}
}

Thanks in advance!

James_Jelinek · August 5, 2015, 1:58pm

Kyle,

Can you suggest a simple grep style script which will return ok, warn, critical so I can parse a log file for a keyword?

If I used ! grep ERROR /path/to/log.file how would I get the OK and Warnings into sensu?

-james

···

On Saturday, July 4, 2015 at 7:31:18 PM UTC-5, Kyle Anderson wrote:

The community check-log is not designed to work like that:
https://github.com/sensu/sensu-community-plugins/blob/af719c4a11a088ad5f601520a95ff1dd09ea31d9/plugins/logging/check-log.rb#L6

You could use any nagios-compliant check-grep style script, or even
KISS with “! grep ERROR /opt/test/logs/testDocker.log”? (only would
ever return ok or warn though)

On Fri, Jul 3, 2015 at 1:47 PM, Romario Lodder romari...@gmail.com wrote:

Hi,

I’m using the check_log to check my logfiles on certain keywords, for example “error”. If a new error is written to the log file, sensu gives me an error. After a couple of seconds, the critical error disappears. How should I configure my check in order to check the log file on a word like ERROR and that the error stays as long the error exists in the log file?

My check looks a bit like this:

{
“checks”: {
“check_log”: {
“command”: “/etc/sensu/plugins/check-log.rb -f /opt/test/logs/testDocker.log -q ‘ERROR’”,
“interval”: 10,
“subscribers”: [“subscriber1”]
}
}
}

Thanks in advance!

Kyle_Anderson · August 5, 2015, 2:42pm

I don't have any good suggestions, the first google hit for "nagios
check grep" is this:

···

On Wed, Aug 5, 2015 at 6:58 AM, James Jelinek <jelinek@gmail.com> wrote:

Kyle,

Can you suggest a simple grep style script which will return ok, warn,
critical so I can parse a log file for a keyword?

If I used ! grep ERROR /path/to/log.file how would I get the OK and Warnings
into sensu?

-james

On Saturday, July 4, 2015 at 7:31:18 PM UTC-5, Kyle Anderson wrote:

The community check-log is not designed to work like that:

https://github.com/sensu/sensu-community-plugins/blob/af719c4a11a088ad5f601520a95ff1dd09ea31d9/plugins/logging/check-log.rb#L6

You could use any nagios-compliant check-grep style script, or even
KISS with "! grep ERROR /opt/test/logs/testDocker.log"? (only would
ever return ok or warn though)

On Fri, Jul 3, 2015 at 1:47 PM, Romario Lodder <romari...@gmail.com> >> wrote:
> Hi,
>
> I'm using the check_log to check my logfiles on certain keywords, for
> example "error". If a new error is written to the log file, sensu gives me
> an error. After a couple of seconds, the critical error disappears. How
> should I configure my check in order to check the log file on a word like
> ERROR and that the error stays as long the error exists in the log file?
>
>
> My check looks a bit like this:
>
> {
> "checks": {
> "check_log": {
> "command": "/etc/sensu/plugins/check-log.rb -f
> /opt/test/logs/testDocker.log -q 'ERROR'",
> "interval": 10,
> "subscribers": ["subscriber1"]
> }
> }
> }
>
> Thanks in advance!

James_Jelinek · August 5, 2015, 5:19pm

I got the check-log.rb to work with the help of the Sensu dev guys.

Here’s the command I run in my definition

/etc/sensu/plugins/check-log.rb -s /tmp/cache/check-log -f /path/to/log.file -q Error -w ‘1’ -c

‘2’"

This will result in critical or warning being issued the first time, but when it runs the check again it will zero out the results because it already read the file and stored it in cache. The fix to keep the alert alive was given by Sean Porter of Sensu and that’s to include “auto_resolve”: false, in the check definition so the critical or warning alert persists.

So far this is working in my environment. My bigger issue is, my log file names change every day. So I’m trying to figure out a way to symlink the latest log file to /var/log/latestimport.log and have this happen every day.

Any thoughts on dynamic symlinking based off of mtime?

···

On Aug 5, 2015, at 9:42 AM, Kyle Anderson kyle@xkyle.com wrote:

I don’t have any good suggestions, the first google hit for “nagios
check grep” is this:
https://exchange.nagios.org/directory/Plugins/Operating-Systems/Linux/check-grep-command-output/details

On Wed, Aug 5, 2015 at 6:58 AM, James Jelinek jelinek@gmail.com wrote:

Kyle,

Can you suggest a simple grep style script which will return ok, warn,
critical so I can parse a log file for a keyword?

If I used ! grep ERROR /path/to/log.file how would I get the OK and Warnings
into sensu?

-james

On Saturday, July 4, 2015 at 7:31:18 PM UTC-5, Kyle Anderson wrote:

The community check-log is not designed to work like that:

https://github.com/sensu/sensu-community-plugins/blob/af719c4a11a088ad5f601520a95ff1dd09ea31d9/plugins/logging/check-log.rb#L6

You could use any nagios-compliant check-grep style script, or even
KISS with “! grep ERROR /opt/test/logs/testDocker.log”? (only would
ever return ok or warn though)

On Fri, Jul 3, 2015 at 1:47 PM, Romario Lodder romari...@gmail.com > > > wrote:

Hi,

I’m using the check_log to check my logfiles on certain keywords, for
example “error”. If a new error is written to the log file, sensu gives me
an error. After a couple of seconds, the critical error disappears. How
should I configure my check in order to check the log file on a word like
ERROR and that the error stays as long the error exists in the log file?

My check looks a bit like this:

{
“checks”: {
“check_log”: {
“command”: “/etc/sensu/plugins/check-log.rb -f
/opt/test/logs/testDocker.log -q ‘ERROR’”,
“interval”: 10,
“subscribers”: [“subscriber1”]
}
}
}

Thanks in advance!

Ethan_Erchinger · August 9, 2015, 5:47pm

`ln -s $(ls -rt /path/to/file.* | tail -n1) /symlink/path`

James_Jelinek · August 9, 2015, 7:12pm

This is exactly what I’m looking for. Thanks! Dynamic symlinking here i come

Appreciate it, Ethan!

···

On Aug 9, 2015, at 12:47 PM, Ethan Erchinger <ethan@erchinger.org> wrote:

`ln -s $(ls -rt /path/to/file.* | tail -n1) /symlink/path`

Laz · February 5, 2016, 10:31am

I have the same issue where the alert is cleared during the next check. As suggested, I tried adding “auto_resolve”: false but still have the same issue.

Do I need to update the check-log.rb file ?.

Thanks for any suggestions.

···

On Wednesday, 5 August 2015 18:19:59 UTC+1, James Jelinek wrote: