Well, finally got an HA Sensu setup using clustered rabbit and redis-sentinel. But right now it seems we have to use HAProxy to pretend there is a single address for everything. In particular Rabbitmq and Redis.
Rabbit it works pretty well since you can write to any rabbit server in the rabbit cluster (assuming we’re using mirrored queus which we did).
But redis is truely a master/slave and with redis-sentinel, it can be somewhat random over time which redis server is master or slave. But there are ways for a redis client to query redis sentinel before it connects and find out which is master. If the sensu consumers of redis could take advantage of that feature we could eliminate haproxy for redis (though you may still need haproxy for talking to redis-sentinel unless the sensu redis clients could do round-robbin access of redis-sentinel servers.)
And while I’m wishing, It would be nice if the rabbit clients also could be given a list of rabbit server IPs / hostnames and would automatically round robin on error so we dodn’t need haproxy for rabbit ehter.
But FYI HA sensu can be done now. Will try to blog about it when I’m less crispy
Rob
Hey Rob,
Any chance u got around to writing anything up on your HA Sensu setup?
-John
···
On Tuesday, September 17, 2013 4:52:54 AM UTC-4, Robert Berger wrote:
Well, finally got an HA Sensu setup using clustered rabbit and redis-sentinel. But right now it seems we have to use HAProxy to pretend there is a single address for everything. In particular Rabbitmq and Redis.
Rabbit it works pretty well since you can write to any rabbit server in the rabbit cluster (assuming we’re using mirrored queus which we did).
But redis is truely a master/slave and with redis-sentinel, it can be somewhat random over time which redis server is master or slave. But there are ways for a redis client to query redis sentinel before it connects and find out which is master. If the sensu consumers of redis could take advantage of that feature we could eliminate haproxy for redis (though you may still need haproxy for talking to redis-sentinel unless the sensu redis clients could do round-robbin access of redis-sentinel servers.)
And while I’m wishing, It would be nice if the rabbit clients also could be given a list of rabbit server IPs / hostnames and would automatically round robin on error so we dodn’t need haproxy for rabbit ehter.
But FYI HA sensu can be done now. Will try to blog about it when I’m less crispy
Rob
Rob, some progress or news about your post 
hope can read this soon, we are interested in your experience!
···
El martes, 17 de septiembre de 2013 05:52:54 UTC-3, Robert Berger escribió:
Well, finally got an HA Sensu setup using clustered rabbit and redis-sentinel. But right now it seems we have to use HAProxy to pretend there is a single address for everything. In particular Rabbitmq and Redis.
Rabbit it works pretty well since you can write to any rabbit server in the rabbit cluster (assuming we’re using mirrored queus which we did).
But redis is truely a master/slave and with redis-sentinel, it can be somewhat random over time which redis server is master or slave. But there are ways for a redis client to query redis sentinel before it connects and find out which is master. If the sensu consumers of redis could take advantage of that feature we could eliminate haproxy for redis (though you may still need haproxy for talking to redis-sentinel unless the sensu redis clients could do round-robbin access of redis-sentinel servers.)
And while I’m wishing, It would be nice if the rabbit clients also could be given a list of rabbit server IPs / hostnames and would automatically round robin on error so we dodn’t need haproxy for rabbit ehter.
But FYI HA sensu can be done now. Will try to blog about it when I’m less crispy
Rob