Load balancing agent connections using TLS

Sensu Go
1

Greetings folks,

I’ve been working on load balancing my agent connections via an Nginx load balancer and have been using @jspaleta 's Jef Practice: Nginx loadbalancer for Sensu Go cluster, but don’t seem to have much luck getting the connection to work correctly. Here’s what I’m presently using:

upstream sensu_agent {
   ip_hash;
   server sensu00.sachshaus.net:8081;
   server sensu01.sachshaus.net:8081;
   server sensu02.sachshaus.net:8081;
}

server {
    listen *:8081;

    location / {
        # redirect all HTTP traffic to sensu_agent_ws loadbalancer defined above:
        proxy_pass https://sensu_agent;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # WebSocket support (nginx 1.4)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

I might be mistakenly thinking that just subbing https for http in the proxy_pass directive would work, but I’m curious if anyone’s got any pointers or anything different that they’re doing, as I’m getting the following:

Jan 12 21:02:04 logs01.sachshaus.net sensu-agent[126175]: {"component":"agent","error":"tls: first record does not look like a TLS handshake","level":"error","msg":"reconnection attempt failed","time":"2021-01-12T21:02:04Z"}

Which leads me to think I’ve not configured Nginx correctly.

1 Like
2

NVM, definitely a misconfig on my end.

1 Like
3

Hi there, @aaronsachs :wave: I hope you’re well!

We have a built-in agent connection load balancing solution coming soon, so let us know how this goes for you and whether a built-in solution would be preferred. :slightly_smiling_face:

1 Like
4

Howdy! Doing well and apparently missed your reply :man_facepalming: . Nginx load balancing seems to be working well, since I’m pushing a bunch of other traffic through it. I’m curious about the upcoming feature, though. Are y’all going to randomizing the connection strings from the agent configs, or taking a different approach?

5

Sensu Go 6.3.0 was released on May 26th and features new built-in rate limiting.

https://docs.sensu.io/sensu-go/latest/release-notes/#630-release-notes

For more information see the Sensu Backend --agent-burst-limit and --agent-rate-limit flags.

I hope this helps!