Hi, we have noticed some weird behavior of sensu-go regarding silences.
We have three silences in place. One of them is already expired but still shows via API as well as in the UI. Moreover the test silences are created via the web interface while the kube-pods silence was created via API.
While the test silences show perfectly fine in the web UI, the kube-pods silence has it’s point in time running backwards in time when hovering over the value (see screenshots, compare with remaining time of test silences).
In contrast to this, the expire times behave weirdly for the test silences when queried via API whereas the kube-pods silence is perfectly fine (see code snipped; executions are top to bottom, old to new).
Any idea what is causing this?
$ sensuctl silenced list
Name Subscription Check Begin Expire ExpireOnResolve Creator Reason Namespace
──────────────────── ────────────────── ─────── ────────────────────── ────────── ───────────────── ────────── ────────────────────────────────── ───────────
*:test test 27 Aug 20 12:00 CEST -1h13m2s false admin test rzneo
*:test2 test2 27 Aug 20 11:00 CEST 30m17s false admin rzneo
entity:kube-pods:* entity:kube-pods 27 Aug 20 06:30 CEST 30m0s false silencer daily silence on cluster startup rzneo
$ sensuctl silenced list
Name Subscription Check Begin Expire ExpireOnResolve Creator Reason Namespace
──────────────────── ────────────────── ─────── ────────────────────── ─────────── ───────────────── ────────── ────────────────────────────────── ───────────
*:test test 27 Aug 20 12:00 CEST -1h12m52s false admin test rzneo
*:test2 test2 27 Aug 20 11:00 CEST 30m27s false admin rzneo
entity:kube-pods:* entity:kube-pods 27 Aug 20 06:30 CEST 30m0s false silencer daily silence on cluster startup rzneo
$ sensuctl silenced list
Name Subscription Check Begin Expire ExpireOnResolve Creator Reason Namespace
──────────────────── ────────────────── ─────── ────────────────────── ─────────── ───────────────── ────────── ────────────────────────────────── ───────────
*:test test 27 Aug 20 12:00 CEST -1h11m52s false admin test rzneo
*:test2 test2 27 Aug 20 11:00 CEST 31m27s false admin rzneo
entity:kube-pods:* entity:kube-pods 27 Aug 20 06:30 CEST 30m0s false silencer daily silence on cluster startup rzneo