My team is looking to replace an aging, proprietary alerting system with a new one assembled primarily from open-source components. Currently, we are looking at using OpenTSDB for data storage, scollector for collecting system-level metrics and a homegrown RESTful API for stakeholders to upload ad hoc data series.
The next component to choose is the event detection engine. The workflow we have envisioned is that all events are detected via queries run on an interval against the OpenTSDB data source (assuming it is efficient enough). We were interested in Bosun, but had concerns about its scalability. We are now looking at Sensu, but we are not sure if the Sensu paradigm meshes well with the workflow we have in mind.
Are we barking up the wrong tree here? Does Sensu make sense for us? If not, does anyone have a recommendation for another system to investigate?