The server-scheduled checks are staggered across the first 30 seconds after server startup. You can look at the log to see the exact run timestamps. Potentially the code could be extended to allow specification of a initial delay for each check, but then there is an increasing cost to restarting the sensu-server (since restarting regularly might prevent some checks from running at all).
If you want complete flexibility, you can forgo scheduling checks through the server. One way would be to set ‘publish’ to false for each check, and trigger them manually via the API as needed. Or, you could write an external script that looped through and used the push mechanism to push warnings into sensu via TCP or UDP. This approach might be best. Lastly, you can always spread the checks across different servers with some clever subscription use or standalone checks.
For heavier calls like rds-describe-instances, you might need to get a little creative, but sensu itself is fine with issuing check requests. We handle thousands of ping checks every few minutes, which works out since the checks are all lightweight.
Hope that helps!
On Friday, May 31, 2013 11:51:37 AM UTC-7, Micah Hoffmann wrote:
Hello I am running into a issue where my sensu server is becoming loaded because I have not devised a plan to stagger my checks.
I have 12 .json check files that each run 22 of the same ruby command. I believe they all fire off at the same time…! 264 commands running rds-describe-instances -_- so it is very heavy.
Has anyone out there found a way to configure the sensu scheduler to stagger the checks? To clarify, I’m not talking about the occurrences and intervals within a check, im talking higher level.