Sensu - AMQP - Graphite


#1

Hi Guys,

I’m trying to use Sensu -> RabbitMQ -> Graphite

I have this working, or almost… the issue I have that each Sensu-client create your own Exchange, into Rabbit and this don’t match with the Carbon…

Into carbon-cache.py I have:

[cache]

LOCAL_DATA_DIR = /opt/graphite/storage/whisper

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = rabbitmq.mgmt-int

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = PASS

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

Sensu handler:

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

And from RabbitMQ side

Virtual host
Name
Type
Policy
Parameters
Message rate in
Message rate out
/
(AMQP default)
direct
D
/
amq.direct
direct
D
/
amq.fanout
fanout
D
/
amq.headers
headers
D
/
amq.match
headers
D
/
amq.rabbitmq.log
topic
D
/
amq.rabbitmq.trace
topic
D
/
amq.topic
topic
D
/sensu
(AMQP default)
direct
D
/sensu
Amazon
fanout
/sensu
DMTLA
fanout
/sensu
SAO-Web
fanout
/sensu
amq.direct
direct
D
/sensu
amq.fanout
fanout
D
/sensu
amq.headers
headers
D
/sensu
amq.match
headers
D
/sensu
amq.rabbitmq.trace
topic
D
/sensu
amq.topic
topic
D
/sensu
ci_server
fanout
/sensu
gateway_server
fanout
/sensu
gen-int
fanout
/sensu
keepalives
direct
0.20/s
0.20/s
/sensu
liferay_server
fanout
/sensu
mgmt-int
fanout
/sensu
puppet_server
fanout
/sensu
qa-int
fanout
/sensu
results
direct
0.00/s
0.00/s
/sensu
varnish_server
fanout

You can see that METRICS exchange never be created but all this others yes… this come from:

“subscribers”: [ “Amazon” ]

Definition into VMSTATS check metrics

Some advice will be welcome.

Regards,

Ale


#2

Hi,

create a graphite handler

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

and change your checks to use this handler.

i have it working.

···

On Thursday, August 15, 2013 11:15:59 PM UTC+1, Alejandro Ferrari wrote:

Hi Guys,

I’m trying to use Sensu -> RabbitMQ -> Graphite

I have this working, or almost… the issue I have that each Sensu-client create your own Exchange, into Rabbit and this don’t match with the Carbon…

Into carbon-cache.py I have:

[cache]

LOCAL_DATA_DIR = /opt/graphite/storage/whisper

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = rabbitmq.mgmt-int

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = PASS

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

Sensu handler:

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

And from RabbitMQ side

Virtual host
Name
Type
Policy
Parameters
Message rate in
Message rate out
/
(AMQP default)
direct
D
/
amq.direct
direct
D
/
amq.fanout
fanout
D
/
amq.headers
headers
D
/
amq.match
headers
D
/
amq.rabbitmq.log
topic
D
/
amq.rabbitmq.trace
topic
D
/
amq.topic
topic
D
/sensu
(AMQP default)
direct
D
/sensu
Amazon
fanout
/sensu
DMTLA
fanout
/sensu
SAO-Web
fanout
/sensu
amq.direct
direct
D
/sensu
amq.fanout
fanout
D
/sensu
amq.headers
headers
D
/sensu
amq.match
headers
D
/sensu
amq.rabbitmq.trace
topic
D
/sensu
amq.topic
topic
D
/sensu
ci_server
fanout
/sensu
gateway_server
fanout
/sensu
gen-int
fanout
/sensu
keepalives
direct
0.20/s
0.20/s
/sensu
liferay_server
fanout
/sensu
mgmt-int
fanout
/sensu
puppet_server
fanout
/sensu
qa-int
fanout
/sensu
results
direct
0.00/s
0.00/s
/sensu
varnish_server
fanout

You can see that METRICS exchange never be created but all this others yes… this come from:

“subscribers”: [ “Amazon” ]

Definition into VMSTATS check metrics

Some advice will be welcome.

Regards,

Ale


#3

Luis,

This is my setup

[root@monitoring-01 sensu]# cat conf.d/handler_graphite.json

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

[root@monitoring-01 sensu]# cat conf.d/metrics_vmstat.json

{

“checks”: {

“vmstat_metrics”: {

“type”: “metric”,

“handlers”: [“graphite”],

“command”: “/etc/sensu/plugins/vmstat-metrics.rb --scheme stats.:::name:::”,

“interval”: 60,

“subscribers”: [ “Amazon” ]

}

}

}

Do you see something wrong in my setup?

thanks

Alejandro

···

El domingo, 8 de septiembre de 2013 14:44:54 UTC-3, Luis Azedo escribió:

Hi,

create a graphite handler

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

and change your checks to use this handler.

i have it working.

On Thursday, August 15, 2013 11:15:59 PM UTC+1, Alejandro Ferrari wrote:

Hi Guys,

I’m trying to use Sensu -> RabbitMQ -> Graphite

I have this working, or almost… the issue I have that each Sensu-client create your own Exchange, into Rabbit and this don’t match with the Carbon…

Into carbon-cache.py I have:

[cache]

LOCAL_DATA_DIR = /opt/graphite/storage/whisper

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = rabbitmq.mgmt-int

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = PASS

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

Sensu handler:

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

And from RabbitMQ side

Virtual host
Name
Type
Policy
Parameters
Message rate in
Message rate out
/
(AMQP default)
direct
D
/
amq.direct
direct
D
/
amq.fanout
fanout
D
/
amq.headers
headers
D
/
amq.match
headers
D
/
amq.rabbitmq.log
topic
D
/
amq.rabbitmq.trace
topic
D
/
amq.topic
topic
D
/sensu
(AMQP default)
direct
D
/sensu
Amazon
fanout
/sensu
DMTLA
fanout
/sensu
SAO-Web
fanout
/sensu
amq.direct
direct
D
/sensu
amq.fanout
fanout
D
/sensu
amq.headers
headers
D
/sensu
amq.match
headers
D
/sensu
amq.rabbitmq.trace
topic
D
/sensu
amq.topic
topic
D
/sensu
ci_server
fanout
/sensu
gateway_server
fanout
/sensu
gen-int
fanout
/sensu
keepalives
direct
0.20/s
0.20/s
/sensu
liferay_server
fanout
/sensu
mgmt-int
fanout
/sensu
puppet_server
fanout
/sensu
qa-int
fanout
/sensu
results
direct
0.00/s
0.00/s
/sensu
varnish_server
fanout

You can see that METRICS exchange never be created but all this others yes… this come from:

“subscribers”: [ “Amazon” ]

Definition into VMSTATS check metrics

Some advice will be welcome.

Regards,

Ale


#4

Hi,

your setup seems ok.

i remember i had to run “pip install txAMQP” in graphite server in order for the carbon-cache to work.

···

On Monday, September 9, 2013 7:11:56 PM UTC+1, Alejandro Ferrari wrote:

Luis,

This is my setup

[root@monitoring-01 sensu]# cat conf.d/handler_graphite.json

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

[root@monitoring-01 sensu]# cat conf.d/metrics_vmstat.json

{

“checks”: {

“vmstat_metrics”: {

“type”: “metric”,

“handlers”: [“graphite”],

“command”: “/etc/sensu/plugins/vmstat-metrics.rb --scheme stats.:::name:::”,

“interval”: 60,

“subscribers”: [ “Amazon” ]

}

}

}

Do you see something wrong in my setup?

thanks

Alejandro

El domingo, 8 de septiembre de 2013 14:44:54 UTC-3, Luis Azedo escribió:

Hi,

create a graphite handler

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

and change your checks to use this handler.

i have it working.

On Thursday, August 15, 2013 11:15:59 PM UTC+1, Alejandro Ferrari wrote:

Hi Guys,

I’m trying to use Sensu -> RabbitMQ -> Graphite

I have this working, or almost… the issue I have that each Sensu-client create your own Exchange, into Rabbit and this don’t match with the Carbon…

Into carbon-cache.py I have:

[cache]

LOCAL_DATA_DIR = /opt/graphite/storage/whisper

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = rabbitmq.mgmt-int

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = PASS

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

Sensu handler:

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

And from RabbitMQ side

Virtual host
Name
Type
Policy
Parameters
Message rate in
Message rate out
/
(AMQP default)
direct
D
/
amq.direct
direct
D
/
amq.fanout
fanout
D
/
amq.headers
headers
D
/
amq.match
headers
D
/
amq.rabbitmq.log
topic
D
/
amq.rabbitmq.trace
topic
D
/
amq.topic
topic
D
/sensu
(AMQP default)
direct
D
/sensu
Amazon
fanout
/sensu
DMTLA
fanout
/sensu
SAO-Web
fanout
/sensu
amq.direct
direct
D
/sensu
amq.fanout
fanout
D
/sensu
amq.headers
headers
D
/sensu
amq.match
headers
D
/sensu
amq.rabbitmq.trace
topic
D
/sensu
amq.topic
topic
D
/sensu
ci_server
fanout
/sensu
gateway_server
fanout
/sensu
gen-int
fanout
/sensu
keepalives
direct
0.20/s
0.20/s
/sensu
liferay_server
fanout
/sensu
mgmt-int
fanout
/sensu
puppet_server
fanout
/sensu
qa-int
fanout
/sensu
results
direct
0.00/s
0.00/s
/sensu
varnish_server
fanout

You can see that METRICS exchange never be created but all this others yes… this come from:

“subscribers”: [ “Amazon” ]

Definition into VMSTATS check metrics

Some advice will be welcome.

Regards,

Ale


#5

Luis,

Can share with me, the carbon.conf and tell me if you have some debug log? I didn’t find the listener.log but I have the Debug in TRUE.

Thanks for any advice, I don’t understand why this does not work.

Thanks,

Alejandro

···

El jueves, 12 de septiembre de 2013 15:31:34 UTC-3, Luis Azedo escribió:

Hi,

your setup seems ok.

i remember i had to run “pip install txAMQP” in graphite server in order for the carbon-cache to work.

On Monday, September 9, 2013 7:11:56 PM UTC+1, Alejandro Ferrari wrote:

Luis,

This is my setup

[root@monitoring-01 sensu]# cat conf.d/handler_graphite.json

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

[root@monitoring-01 sensu]# cat conf.d/metrics_vmstat.json

{

“checks”: {

“vmstat_metrics”: {

“type”: “metric”,

“handlers”: [“graphite”],

“command”: “/etc/sensu/plugins/vmstat-metrics.rb --scheme stats.:::name:::”,

“interval”: 60,

“subscribers”: [ “Amazon” ]

}

}

}

Do you see something wrong in my setup?

thanks

Alejandro

El domingo, 8 de septiembre de 2013 14:44:54 UTC-3, Luis Azedo escribió:

Hi,

create a graphite handler

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

and change your checks to use this handler.

i have it working.

On Thursday, August 15, 2013 11:15:59 PM UTC+1, Alejandro Ferrari wrote:

Hi Guys,

I’m trying to use Sensu -> RabbitMQ -> Graphite

I have this working, or almost… the issue I have that each Sensu-client create your own Exchange, into Rabbit and this don’t match with the Carbon…

Into carbon-cache.py I have:

[cache]

LOCAL_DATA_DIR = /opt/graphite/storage/whisper

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = rabbitmq.mgmt-int

AMQP_PORT = 5672

AMQP_VHOST = /sensu

AMQP_USER = sensu

AMQP_PASSWORD = PASS

AMQP_EXCHANGE = metrics

AMQP_METRIC_NAME_IN_BODY = True

Sensu handler:

{

“handlers”: {

“graphite”: {

“type”: “amqp”,

“exchange”: {

“type”: “topic”,

“name”: “metrics”,

“durable”: “true”

},

“mutator”: “only_check_output”

}

}

}

And from RabbitMQ side

Virtual host
Name
Type
Policy
Parameters
Message rate in
Message rate out
/
(AMQP default)
direct
D
/
amq.direct
direct
D
/
amq.fanout
fanout
D
/
amq.headers
headers
D
/
amq.match
headers
D
/
amq.rabbitmq.log
topic
D
/
amq.rabbitmq.trace
topic
D
/
amq.topic
topic
D
/sensu
(AMQP default)
direct
D
/sensu
Amazon
fanout
/sensu
DMTLA
fanout
/sensu
SAO-Web
fanout
/sensu
amq.direct
direct
D
/sensu
amq.fanout
fanout
D
/sensu
amq.headers
headers
D
/sensu
amq.match
headers
D
/sensu
amq.rabbitmq.trace
topic
D
/sensu
amq.topic
topic
D
/sensu
ci_server
fanout
/sensu
gateway_server
fanout
/sensu
gen-int
fanout
/sensu
keepalives
direct
0.20/s
0.20/s
/sensu
liferay_server
fanout
/sensu
mgmt-int
fanout
/sensu
puppet_server
fanout
/sensu
qa-int
fanout
/sensu
results
direct
0.00/s
0.00/s
/sensu
varnish_server
fanout

You can see that METRICS exchange never be created but all this others yes… this come from:

“subscribers”: [ “Amazon” ]

Definition into VMSTATS check metrics

Some advice will be welcome.

Regards,

Ale


#6

Ok, guys I found the issue, into Centos 6.3 or similar, if use the version of txAMQP from the repo, is really old, after install this from Pip all start to work perfect!

Sorry if I don’t see this before, just I don’t identify that was different versions from repo vs pip.

Thanks!

Ale


#7