Hello
I’m trying to setup a three node Sensu Go cluster using Sensu Go 6.6 on Red Hat Enterprise Linux 8.3 servers. I have no firewall between the nodes (not the same subnet).
I have a working “test-cluster” using Sensu Go 6.3 and I have pretty much replicated the settings in backend.yml to my 6.6 setup.
Starting all nodes works fine.
When i Run: sudo sensu-backend init
I receive the following error message,
{“component”:“cmd”,“level”:“info”,“msg”:“attempting to connect to etcd server: https://10.0.0.1:2379”,“time”:“2022-03-02T08:32:48+01:00”}
{“component”:“cmd”,“level”:“error”,“msg”:“error connecting to etcd endpoint: context deadline exceeded”,“time”:“2022-03-02T08:32:53+01:00”}
{“component”:“sensu-enterprise”,“error”:“no etcd endpoints are available or cluster is unhealthy”,“level”:“fatal”,“msg”:“error executing sensu-backend”,“time”:“2022-03-02T08:32:53+01:00”}
sudo journalctl --follow --unit sensu-backend
Error log.
{“cache_version”:“v1”,“component”:“cache”,“level”:“debug”,“msg”:“rebuilding the cache for resource type *v2.Silenced”,“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/interceptor.go:182”,“msg”:“request stats”,“start time”:“2022-03-02T08:36:05+01:00”,“time spent”:“60.529µs”,“remote”:“10.0.0.157520”,“response type”:"/etcdserverpb.KV/Range",“request count”:0,“request size”:43,“response count”:0,“response size”:28,“request content”:“key:”/sensu.io/silenced/" range_end:"/sensu.io/silencee" serializable:true “,“time”:“2022-03-02T08:36:05+01:00”}
{“cache_version”:“v2”,“component”:“cache”,“level”:“debug”,“msg”:“rebuilding the cache for resource type *v3.EntityConfig”,“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/interceptor.go:182”,“msg”:“request stats”,“start time”:“2022-03-02T08:36:05+01:00”,“time spent”:“45.781µs”,“remote”:“10.0.0.157520”,“response type”:”/etcdserverpb.KV/Range",“request count”:0,“request size”:55,“response count”:0,“response size”:28,“request content”:“key:”/sensu.io/entity_configs/" range_end:"/sensu.io/entity_configt" serializable:true “,“time”:“2022-03-02T08:36:05+01:00”}
{“cache_version”:“v1”,“component”:“cache”,“level”:“debug”,“msg”:“rebuilding the cache for resource type *v2.Namespace”,“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/interceptor.go:182”,“msg”:“request stats”,“start time”:“2022-03-02T08:36:05+01:00”,“time spent”:“35.062µs”,“remote”:“10.0.0.157520”,“response type”:”/etcdserverpb.KV/Range",“request count”:0,“request size”:47,“response count”:0,“response size”:28,“request content”:“key:”/sensu.io/namespaces/" range_end:"/sensu.io/namespacet" serializable:true “,“time”:“2022-03-02T08:36:05+01:00”}
{“cache_version”:“v2”,“component”:“cache”,“level”:“debug”,“msg”:“rebuilding the cache for resource type *v3.EntityConfig”,“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/interceptor.go:182”,“msg”:“request stats”,“start time”:“2022-03-02T08:36:05+01:00”,“time spent”:“30.164µs”,“remote”:“10.0.0.157520”,“response type”:”/etcdserverpb.KV/Range",“request count”:0,“request size”:55,“response count”:0,“response size”:28,“request content”:“key:”/sensu.io/entity_configs/" range_end:"/sensu.io/entity_configt" serializable:true “,“time”:“2022-03-02T08:36:05+01:00”}
{“backend_id”:“3d4b3900-089e-445f-97c2-2f1ea35c61a9”,“component”:“metricsd”,“level”:“debug”,“msg”:“metricsd heartbeat”,“name”:“entity_metrics”,“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/interceptor.go:182”,“msg”:“request stats”,“start time”:“2022-03-02T08:36:05+01:00”,“time spent”:“1.151711ms”,“remote”:“10.0.0.157520”,“response type”:”/etcdserverpb.KV/Range",“request count”:0,“request size”:78,“response count”:1,“response size”:128,“request content”:“key:”/sensu.io/rings/metricsd/backends/items/3d4b3900-089e-445f-97c2-2f1ea35c61a9" ",“time”:“2022-03-02T08:36:05+01:00”}
{“component”:“etcd”,“level”:“debug”,“caller”:“v3rpc/lease.go:118”,“msg”:“failed to receive lease keepalive request from gRPC stream”,“error”:“rpc error: code = Canceled desc = context canceled”,“time”:“2022-03-02T08:36:05+01:00”}
anyone got any ideas?
Best regards
Lucas