# Filter seems is not working for the events that send to pagerduty

**URL:** <https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537>\
**Category:** Sensu Go\
**Created:** [April 15, 2021, 8:07pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537 "2021-04-15T20:07:24Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![melw](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@melw](https://discourse.sensu.io/u/melw)\
**Post date:** [April 15, 2021, 8:07pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/1 "2021-04-15T20:07:25Z")

</div>

I have a check that integrate with pagerduty and will page me once there are more than three Criticals events, with using the sensu-plugin/handler-pagerduty.rb ([sensu-plugins-pagerduty/handler-pagerduty.rb at master · sensu-plugins/sensu-plugins-pagerduty · GitHub](https://github.com/sensu-plugins/sensu-plugins-pagerduty/blob/master/bin/handler-pagerduty.rb)) The event mapped the into sensugo with [sensu-plugin/sensu-handler.rb at master · sensu-plugins/sensu-plugin · GitHub](https://github.com/sensu-plugins/sensu-plugin/blob/master/lib/sensu-handler.rb#L17)

Now the issue is I got paged for even a single error event(there are mainly two issues in this case. First, the occurrences filter here is not working, and second issue is, it couldn’t override the non-critical event into a correct event severity. i.e. when the event send to pagerduty, all kinds of events treated at a Critical level. For the second issue I find this plugin file could solve the issue [sensu-plugins-pagerduty/mutator-pagerduty-priority-override.rb at master · sensu-plugins/sensu-plugins-pagerduty · GitHub](https://github.com/sensu-plugins/sensu-plugins-pagerduty/blob/master/bin/mutator-pagerduty-priority-override.rb), but occurrences filter not working to page us only if above 3 events is still a problem at this moment)

Could you please have a look at the issue, this is weird because the occurrences filter is working for email and slack alert handlers, only when sending pager duty, it is not working. Is there a way can solve this issue?

**The event json file:**

```auto
{
  "check": {
    "command": "echo testing..... && /4xx-error-rates.py --endpoint \"%\" --status_code \"401\"",
    "handlers": [
      "pd-test"
    ],
    "high_flap_threshold": 0,
    "interval": 300,
    "low_flap_threshold": 0,
    "publish": true,
    "runtime_assets": null,
    "subscriptions": [
      "runner",
      "runner-stg"
    ],
    "proxy_entity_name": "api-checker",
    "check_hooks": null,
    "stdin": false,
    "subdue": null,
    "ttl": 0,
    "timeout": 0,
    "round_robin": true,
    "duration": 0.10803472,
    "executed": 1618516354,
    "history": [
      {
        "status": 127,
        "executed": 1618511247
      },
      {
        "status": 127,
        "executed": 1618511547
      },
      {
        "status": 0,
        "executed": 1618511547
      },
      {
        "status": 127,
        "executed": 1618511848
      },
      {
        "status": 127,
        "executed": 1618512148
      },
      {
        "status": 127,
        "executed": 1618512449
      },
      {
        "status": 127,
        "executed": 1618512749
      },
      {
        "status": 127,
        "executed": 1618513049
      },
      {
        "status": 127,
        "executed": 1618513350
      },
      {
        "status": 127,
        "executed": 1618513650
      },
      {
        "status": 127,
        "executed": 1618513951
      },
      {
        "status": 127,
        "executed": 1618514251
      },
      {
        "status": 127,
        "executed": 1618514552
      },
      {
        "status": 127,
        "executed": 1618514852
      },
      {
        "status": 127,
        "executed": 1618515152
      },
      {
        "status": 127,
        "executed": 1618515453
      },
      {
        "status": 127,
        "executed": 1618515753
      },
      {
        "status": 127,
        "executed": 1618516054
      },
      {
        "status": 127,
        "executed": 1618516056
      },
      {
        "status": 0,
        "executed": 1618516056
      },
      {
        "status": 127,
        "executed": 1618516354
      }
    ],
    "issued": 1618516354,
    "output": "testing.....\nsh: /4xx-error-rates.py: No such file or directory\n",
    "state": "failing",
    "status": 127,
    "total_state_change": 20,
    "last_ok": 1618516056,
    "occurrences": 1,
    "occurrences_watermark": 1,
    "output_metric_format": "",
    "output_metric_handlers": null,
    "env_vars": null,
    "metadata": {
      "name": "apiv3_errors_401",
      "namespace": "stg",
      "labels": {
        "sensu.io/managed_by": "sensuctl",
        "notification": "APIV3 403 Error Rate"
      }
    },
    "secrets": null,
    "is_silenced": false,
    "scheduler": "etcd"
  },
  "entity": {
    "entity_class": "proxy",
    "system": {
      "network": {
        "interfaces": null
      },
      "libc_type": "",
      "vm_system": "",
      "vm_role": "",
      "cloud_provider": "",
      "processes": null
    },
    "subscriptions": [
      "entity:api-checker"
    ],
    "last_seen": 0,
    "deregister": false,
    "deregistration": {},
    "metadata": {
      "name": "api-checker",
      "namespace": "stg"
    },
    "sensu_agent_version": ""
  },
  "id": "0cc06dd9-959d-4a7c-be2b-2df4986aa757",
  "metadata": {
    "namespace": "stg"
  },
  "sequence": 1,
  "timestamp": 1618516354
}

```

**The handler and the filter configuration:**

```auto
type: Handler
api_version: core/v2
metadata:
  name: 'pd-test'
  namespace: stg
spec:
  type: set
  filters:
   - is_incident
   - not_silenced
   - emb-occurrences_3_filter
  handlers:
   - pagerduty_api_sre

---
type: Handler
api_version: core/v2
metadata:
  name: 'pagerduty_api_sre'
  namespace: stg
spec:
  command: "/opt/sensu-plugins-ruby/embedded/bin/handler-pagerduty.rb -j pagerduty_api_sre --map-go-event-into-ruby"
  type: pipe
  filters:
  - is_incident
  - not_silenced

```

**Filter configuration**

```auto
---
  type: EventFilter
  api_version: core/v2
  metadata:
    name: "emb-occurrences_3_filter"
    namespace: stg
  spec:
    action: allow
    expressions:
    - event.check.occurrences >= 3

```

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [April 16, 2021, 2:37am UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/2 "2021-04-16T02:37:14Z")

</div>

Hey,  
I’ll try to reproduce this as soon as I can.  
The fact that you say its working for email with the same filter is very interesting.  
Can you provide the handler definition for the thing that is working as you expect so I can compare?

---

<div class="post-metadata">

**Author:** ![melw](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@melw](https://discourse.sensu.io/u/melw)\
**Post date:** [April 16, 2021, 5:12pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/3 "2021-04-16T17:12:12Z")

</div>

Hi,

Looks like the filter is also not working now ☹ , it worked once when I trigger the execute button to generate a testing error event from UI. I’m using this plugin for email handler

> <https://github.com/sensu-plugins/sensu-plugins-mailer/blob/master/bin/handler-mailer.rb>

I’m so confused why the filter is not working

```auto
---
type: Handler
api_version: core/v2
metadata:
  name: "handler-filter-test"
  namespace: stg
spec:
  type: set
  filters:
  - not_silenced
  - is_incident
  - emb-occurrences_3_filter
  handlers:
  - slack-integration-tests
  - mailer

```

```auto
---
type: Handler
api_version: core/v2
metadata:
  name: 'mailer'
  namespace: stg
spec:
  command: "/opt/sensu-plugins-ruby/embedded/bin/handler-mailer.rb -j test-mail -c html -s '[Sensu]' --map-go-event-into-ruby"
  type: pipe
  filters:
  - is_incident
  - not_silenced

```

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [April 16, 2021, 6:12pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/4 "2021-04-16T18:12:10Z")

</div>

okay,  
well that’s a little less weird.

---

<div class="post-metadata">

**Author:** ![melw](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@melw](https://discourse.sensu.io/u/melw)\
**Post date:** [April 16, 2021, 6:22pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/5 "2021-04-16T18:22:32Z")

</div>

so why the filter with occurrences checking expression doesn’t work in sensu go? Shall I have to use the fatigue\_check asset for implement the occurrences keyword from sensu core?

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [April 16, 2021, 6:24pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/6 "2021-04-16T18:24:14Z")

</div>

i need to play with your filter a bit to understand what’s going on. I’ll look at it today

---

<div class="post-metadata">

**Author:** ![melw](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@melw](https://discourse.sensu.io/u/melw)\
**Post date:** [April 19, 2021, 6:05pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/7 "2021-04-19T18:05:37Z")

</div>

Hi jspaleta,

I doubt this could be some configuration issues in the server side, since we have two systems(dev build and the production one which including multiple namespaces for testing production namespace and a real production namespace). And the same setup in the dev build system is working. Do you have any hint of which configuration could be related with and I can look into the differences of the two systems

---

<div class="post-metadata">

**Author:** ![anatolijd](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/anatolijd/32/507_2.png) [@anatolijd](https://discourse.sensu.io/u/anatolijd)\
**Post date:** [April 21, 2021, 10:57am UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/8 "2021-04-21T10:57:49Z")

</div>

Hi ,

Isn’t it because `pd-test` is a handler set and not a regular filter ?

> Filters specified in a handler set definition have no effect. Filters must be specified in individual handler definitions.

---

<div class="post-metadata">

**Author:** ![melw](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@melw](https://discourse.sensu.io/u/melw)\
**Post date:** [April 21, 2021, 6:42pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/9 "2021-04-21T18:42:05Z")

</div>

oh! This is a nice hint! I just checked, this note is from version 6.0, and I’m using the version 6.2, does this rule applied to the latest version as well?

 ![screenshot-docs.sensu.io-2021.04.21-11_41_37](https://us1.discourse-cdn.com/flex016/uploads/sensu/original/1X/2251e793ecace9a99b2b992763d9455da7b2320f.jpeg)

---

<div class="post-metadata">

**Author:** ![jspaleta](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/jspaleta/32/141_2.png) [@jspaleta](https://discourse.sensu.io/u/jspaleta)\
**Post date:** [April 21, 2021, 7:11pm UTC](https://discourse.sensu.io/t/filter-seems-is-not-working-for-the-events-that-send-to-pagerduty/2537/10 "2021-04-21T19:11:06Z")

</div>

yes

Ref: [https://docs.sensu.io/sensu-go/6.2/observability-pipeline/observe-filter/filters/#event-filter-example-minimum-required-attributes](https://docs.sensu.io/sensu-go/6.2/observability-pipeline/observe-filter/filters/#event-filter-example-minimum-required-attributes)

Small note, you can navigate to the documentation verison that matches the version you are using by using the version dropdown at the top of the docs page.