[amqp] Detected TCP connection failure on local client

Sensu Classic (EOL)
1

Hi,

I followed the instructions detailed here https://sensuapp.org/docs/0.20/ to install sensu on Ubuntu Server 14.04.1

When I try to install and launch the client, it fails to start with the error “[amqp] Detected TCP connection failure”

Here is the full trace of a sensu-client restart :

{“timestamp”:“2015-08-19T09:15:46.992495+0200”,“level”:“warn”,“message”:“received signal”,“signal”:“TERM”}
{“timestamp”:“2015-08-19T09:15:46.992667+0200”,“level”:“warn”,“message”:“stopping”}
{“timestamp”:“2015-08-19T09:15:46.992721+0200”,“level”:“info”,“message”:“completing checks in progress”,“checks_in_progress”:}
{“timestamp”:“2015-08-19T09:15:47.533644+0200”,“level”:“info”,“message”:“closing client tcp and udp sockets”}
{“timestamp”:“2015-08-19T09:15:47.533839+0200”,“level”:“warn”,“message”:“stopping reactor”}
{“timestamp”:“2015-08-19T09:15:48.543522+0200”,“level”:“warn”,“message”:“config file does not exist or is not readable”,“file”:"/etc/sensu/config.json"}
{“timestamp”:“2015-08-19T09:15:48.543762+0200”,“level”:“warn”,“message”:“ignoring config file”,“file”:"/etc/sensu/config.json"}
{“timestamp”:“2015-08-19T09:15:48.543889+0200”,“level”:“warn”,“message”:“loading config files from directory”,“directory”:"/etc/sensu/conf.d"}
{“timestamp”:“2015-08-19T09:15:48.544009+0200”,“level”:“warn”,“message”:“loading config file”,“file”:"/etc/sensu/conf.d/uchiwa.json"}
{“timestamp”:“2015-08-19T09:15:48.544124+0200”,“level”:“warn”,“message”:“loading config file”,“file”:"/etc/sensu/conf.d/api.json"}
{“timestamp”:“2015-08-19T09:15:48.544256+0200”,“level”:“warn”,“message”:“config file applied changes”,“file”:"/etc/sensu/conf.d/api.json",“changes”:{“api”:[null,{“host”:“localhost”,“port”:4567}]}}
{“timestamp”:“2015-08-19T09:15:48.544362+0200”,“level”:“warn”,“message”:“loading config file”,“file”:"/etc/sensu/conf.d/client.json"}
{“timestamp”:“2015-08-19T09:15:48.544474+0200”,“level”:“warn”,“message”:“config file applied changes”,“file”:"/etc/sensu/conf.d/client.json",“changes”:{“client”:[null,{“name”:“test”,“address”:“localhost”,“subscriptions”:[“test”]}]}}
{“timestamp”:“2015-08-19T09:15:48.544593+0200”,“level”:“warn”,“message”:“loading config file”,“file”:"/etc/sensu/conf.d/rabbitmq.json"}
{“timestamp”:“2015-08-19T09:15:48.544734+0200”,“level”:“warn”,“message”:“config file applied changes”,“file”:"/etc/sensu/conf.d/rabbitmq.json",“changes”:{“rabbitmq”:[null,{“ssl”:{“cert_chain_file”:"/etc/sensu/ssl/cert.pem",“private_key_file”:"/etc/sensu/ssl/key.pem"},“host”:“localhost”,“port”:5671,“vhost”:"/sensu",“user”:“sensu”,“password”:“REDACTED”}]}}
{“timestamp”:“2015-08-19T09:15:48.544862+0200”,“level”:“warn”,“message”:“loading config file”,“file”:"/etc/sensu/conf.d/redis.json"}
{“timestamp”:“2015-08-19T09:15:48.544989+0200”,“level”:“warn”,“message”:“config file applied changes”,“file”:"/etc/sensu/conf.d/redis.json",“changes”:{“redis”:[null,{“host”:“localhost”,“port”:6379}]}}
{“timestamp”:“2015-08-19T09:15:48.547478+0200”,“level”:“warn”,“message”:“loading extension files from directory”,“directory”:"/etc/sensu/extensions"}
{“timestamp”:“2015-08-19T09:15:48.628531+0200”,“level”:“error”,“message”:"[amqp] Detected TCP connection failure"}
{“timestamp”:“2015-08-19T09:15:48.628763+0200”,“level”:“warn”,“message”:“reconnecting to transport”}
{“timestamp”:“2015-08-19T09:15:50.703487+0200”,“level”:“error”,“message”:"[amqp] Detected TCP connection failure"}
{“timestamp”:“2015-08-19T09:15:54.766872+0200”,“level”:“error”,“message”:"[amqp] Detected TCP connection failure"}

Any idea on where to look next ?

Thanks.

2

It sounds like a rabbitmq ssl issue (probably). Is rabbitmq configured
with the same corresponding ssl configuration that sensu has? Try
looking at the rabbitmq logs.

···

On Wed, Aug 19, 2015 at 12:22 AM, Rodolphe Beck <rodolphe.beck@gmail.com> wrote:

Hi,

I followed the instructions detailed here https://sensuapp.org/docs/0.20/ to
install sensu on Ubuntu Server 14.04.1

When I try to install and launch the client, it fails to start with the
error "[amqp] Detected TCP connection failure"

Here is the full trace of a sensu-client restart :

{"timestamp":"2015-08-19T09:15:46.992495+0200","level":"warn","message":"received
signal","signal":"TERM"}
{"timestamp":"2015-08-19T09:15:46.992667+0200","level":"warn","message":"stopping"}
{"timestamp":"2015-08-19T09:15:46.992721+0200","level":"info","message":"completing
checks in progress","checks_in_progress":}
{"timestamp":"2015-08-19T09:15:47.533644+0200","level":"info","message":"closing
client tcp and udp sockets"}
{"timestamp":"2015-08-19T09:15:47.533839+0200","level":"warn","message":"stopping
reactor"}
{"timestamp":"2015-08-19T09:15:48.543522+0200","level":"warn","message":"config
file does not exist or is not readable","file":"/etc/sensu/config.json"}
{"timestamp":"2015-08-19T09:15:48.543762+0200","level":"warn","message":"ignoring
config file","file":"/etc/sensu/config.json"}
{"timestamp":"2015-08-19T09:15:48.543889+0200","level":"warn","message":"loading
config files from directory","directory":"/etc/sensu/conf.d"}
{"timestamp":"2015-08-19T09:15:48.544009+0200","level":"warn","message":"loading
config file","file":"/etc/sensu/conf.d/uchiwa.json"}
{"timestamp":"2015-08-19T09:15:48.544124+0200","level":"warn","message":"loading
config file","file":"/etc/sensu/conf.d/api.json"}
{"timestamp":"2015-08-19T09:15:48.544256+0200","level":"warn","message":"config
file applied
changes","file":"/etc/sensu/conf.d/api.json","changes":{"api":[null,{"host":"localhost","port":4567}]}}
{"timestamp":"2015-08-19T09:15:48.544362+0200","level":"warn","message":"loading
config file","file":"/etc/sensu/conf.d/client.json"}
{"timestamp":"2015-08-19T09:15:48.544474+0200","level":"warn","message":"config
file applied
changes","file":"/etc/sensu/conf.d/client.json","changes":{"client":[null,{"name":"test","address":"localhost","subscriptions":["test"]}]}}
{"timestamp":"2015-08-19T09:15:48.544593+0200","level":"warn","message":"loading
config file","file":"/etc/sensu/conf.d/rabbitmq.json"}
{"timestamp":"2015-08-19T09:15:48.544734+0200","level":"warn","message":"config
file applied
changes","file":"/etc/sensu/conf.d/rabbitmq.json","changes":{"rabbitmq":[null,{"ssl":{"cert_chain_file":"/etc/sensu/ssl/cert.pem","private_key_file":"/etc/sensu/ssl/key.pem"},"host":"localhost","port":5671,"vhost":"/sensu","user":"sensu","password":"REDACTED"}]}}
{"timestamp":"2015-08-19T09:15:48.544862+0200","level":"warn","message":"loading
config file","file":"/etc/sensu/conf.d/redis.json"}
{"timestamp":"2015-08-19T09:15:48.544989+0200","level":"warn","message":"config
file applied
changes","file":"/etc/sensu/conf.d/redis.json","changes":{"redis":[null,{"host":"localhost","port":6379}]}}
{"timestamp":"2015-08-19T09:15:48.547478+0200","level":"warn","message":"loading
extension files from directory","directory":"/etc/sensu/extensions"}
{"timestamp":"2015-08-19T09:15:48.628531+0200","level":"error","message":"[amqp]
Detected TCP connection failure"}
{"timestamp":"2015-08-19T09:15:48.628763+0200","level":"warn","message":"reconnecting
to transport"}
{"timestamp":"2015-08-19T09:15:50.703487+0200","level":"error","message":"[amqp]
Detected TCP connection failure"}
{"timestamp":"2015-08-19T09:15:54.766872+0200","level":"error","message":"[amqp]
Detected TCP connection failure"}

Any idea on where to look next ?

Thanks.

3

It will depend on how you setup SSL on rabbitmq and sensu.
The error message hints that rabbitmq is not configured to use the
same cacert that was used to sign whatever sensu is using:
https://www.rabbitmq.com/ssl.html

···

On Wed, Aug 19, 2015 at 8:11 AM, Rodolphe Beck <rodolphe.beck@gmail.com> wrote:

Hi,

I looked at the log, and I have a ssl handshake error :

=ERROR REPORT==== 19-Aug-2015::17:08:58 ===
Error on AMQP connection <0.621.0>:
{ssl_upgrade_error,{tls_alert,"unknown ca"}}

How can I correct it please ?

2015-08-19 16:53 GMT+02:00 Kyle Anderson <kyle@xkyle.com>:

It sounds like a rabbitmq ssl issue (probably). Is rabbitmq configured
with the same corresponding ssl configuration that sensu has? Try
looking at the rabbitmq logs.

On Wed, Aug 19, 2015 at 12:22 AM, Rodolphe Beck <rodolphe.beck@gmail.com> >> wrote:
> Hi,
>
> I followed the instructions detailed here
> https://sensuapp.org/docs/0.20/ to
> install sensu on Ubuntu Server 14.04.1
>
> When I try to install and launch the client, it fails to start with the
> error "[amqp] Detected TCP connection failure"
>
> Here is the full trace of a sensu-client restart :
>
>
> {"timestamp":"2015-08-19T09:15:46.992495+0200","level":"warn","message":"received
> signal","signal":"TERM"}
>
> {"timestamp":"2015-08-19T09:15:46.992667+0200","level":"warn","message":"stopping"}
>
> {"timestamp":"2015-08-19T09:15:46.992721+0200","level":"info","message":"completing
> checks in progress","checks_in_progress":}
>
> {"timestamp":"2015-08-19T09:15:47.533644+0200","level":"info","message":"closing
> client tcp and udp sockets"}
>
> {"timestamp":"2015-08-19T09:15:47.533839+0200","level":"warn","message":"stopping
> reactor"}
>
> {"timestamp":"2015-08-19T09:15:48.543522+0200","level":"warn","message":"config
> file does not exist or is not readable","file":"/etc/sensu/config.json"}
>
> {"timestamp":"2015-08-19T09:15:48.543762+0200","level":"warn","message":"ignoring
> config file","file":"/etc/sensu/config.json"}
>
> {"timestamp":"2015-08-19T09:15:48.543889+0200","level":"warn","message":"loading
> config files from directory","directory":"/etc/sensu/conf.d"}
>
> {"timestamp":"2015-08-19T09:15:48.544009+0200","level":"warn","message":"loading
> config file","file":"/etc/sensu/conf.d/uchiwa.json"}
>
> {"timestamp":"2015-08-19T09:15:48.544124+0200","level":"warn","message":"loading
> config file","file":"/etc/sensu/conf.d/api.json"}
>
> {"timestamp":"2015-08-19T09:15:48.544256+0200","level":"warn","message":"config
> file applied
>
> changes","file":"/etc/sensu/conf.d/api.json","changes":{"api":[null,{"host":"localhost","port":4567}]}}
>
> {"timestamp":"2015-08-19T09:15:48.544362+0200","level":"warn","message":"loading
> config file","file":"/etc/sensu/conf.d/client.json"}
>
> {"timestamp":"2015-08-19T09:15:48.544474+0200","level":"warn","message":"config
> file applied
>
> changes","file":"/etc/sensu/conf.d/client.json","changes":{"client":[null,{"name":"test","address":"localhost","subscriptions":["test"]}]}}
>
> {"timestamp":"2015-08-19T09:15:48.544593+0200","level":"warn","message":"loading
> config file","file":"/etc/sensu/conf.d/rabbitmq.json"}
>
> {"timestamp":"2015-08-19T09:15:48.544734+0200","level":"warn","message":"config
> file applied
>
> changes","file":"/etc/sensu/conf.d/rabbitmq.json","changes":{"rabbitmq":[null,{"ssl":{"cert_chain_file":"/etc/sensu/ssl/cert.pem","private_key_file":"/etc/sensu/ssl/key.pem"},"host":"localhost","port":5671,"vhost":"/sensu","user":"sensu","password":"REDACTED"}]}}
>
> {"timestamp":"2015-08-19T09:15:48.544862+0200","level":"warn","message":"loading
> config file","file":"/etc/sensu/conf.d/redis.json"}
>
> {"timestamp":"2015-08-19T09:15:48.544989+0200","level":"warn","message":"config
> file applied
>
> changes","file":"/etc/sensu/conf.d/redis.json","changes":{"redis":[null,{"host":"localhost","port":6379}]}}
>
> {"timestamp":"2015-08-19T09:15:48.547478+0200","level":"warn","message":"loading
> extension files from directory","directory":"/etc/sensu/extensions"}
>
> {"timestamp":"2015-08-19T09:15:48.628531+0200","level":"error","message":"[amqp]
> Detected TCP connection failure"}
>
> {"timestamp":"2015-08-19T09:15:48.628763+0200","level":"warn","message":"reconnecting
> to transport"}
>
> {"timestamp":"2015-08-19T09:15:50.703487+0200","level":"error","message":"[amqp]
> Detected TCP connection failure"}
>
> {"timestamp":"2015-08-19T09:15:54.766872+0200","level":"error","message":"[amqp]
> Detected TCP connection failure"}
>
>
> Any idea on where to look next ?
>
> Thanks.

--
Cordialement,
Rodolphe Beck
rodolphe.beck@gmail.com