Let me discuss one scenario. Lets I am monitoring a URL e.g main page of any app. That page is returning 200 or OK but there are some request which are may return 500, 401 etc. If there are more than 20 URLs being called from one page, do I have write checks for all URLs to monitor everyone individually?
Similarly Java app is running its self so check-process.rb will return OK but if it is facing some problem due to some exception or error.
How can we handle such use cases?
Hey!
You know that’s a really deep question. There are probably several ways to to application exceptions into events depending on the way your application is designed.
Application log monitoring using a check
If this is an application you can’t modify, you can use a Sensu check to monitor the error logs being produced by the application and/or the web server. This will even work with containerized application when using Sensu agent as a sidecar with access to the application logs.
Lightweight APM using Sensu agent events API
If you can modify the application, you can use the Sensu agent’s events api tcp socket as a lightweight customizable APM platform. You can instrument your application to generate Sensu events as json strings and send them to a running Sensu agent using the agent’s event api. The agent will forward the event on to the Sensu backend as if it were an event generated by a check. Using this pattern you can instrument your code to emit Sensu events for any condition you are concerned about and make it appear as if the agent was checking on your application. The trade-off to using the agent’s events api is you are limited to using some of the agent’s settings, such as the entity and I believe the namespace. If you need full control over the events you want to emit from your application you can access the Sensu backend API directly.
For Sensu Classic users who are migrating, the agent events API is essentially a modernized client socket, serving the same functional role but making using Sensu Go’s event data model.
APM using the Sensu backend API
If you need full authentication and RBAC controls, you can instrument your application to authenticate against the Sensu backend API and populate events directly using a dedicated user role that makes sense for your applications. This is more sophisticated pattern than the agent events api, as it gives you full power of the Sensu API model. There’s room here for the community to generate application language specific libraries to help abstract the Sensu Go backend API interactions to streamline the process.
I hope this helps, there’s a lot of room to explore how to best use the Sensu Go apis as a customizable APM platform.
Thanks for your reply. It was very illuminating. Currently we are unable to change our application modules. Is there any check which monitor log files or do i have to write my own?
You should be able to use the Ruby check-log.rb check from sensu-plugins-logs asset.
The check has a lot of options, but the basic pattern goes like this:
/opt/sensu-plugins-ruby/embedded/bin/check-log.rb -f /var/log/messages -q error
where you set the file you want to watch with -f and then the search pattern with -q
The check by default will hold state in /var/cache/check-log/
You’ll need to make sure the sensu user has permission to read the log files.