I’m seeing these intermittent (about once per day or so?) errors in different checks, like this:
ALERT - [elasticsearch02/elasticsearch_cluster_status_check] - /usr/lib/ruby/1.9.1/rubygems/version.rb:146:in
include': undefined methodappend_features’ for Comparable:Module (NoMethodError) from /usr/lib/ruby/1.9.1/rubygems/version.rb:146:in
<class:version>' from /usr/lib/ruby/1.9.1/rubygems/version.rb:143:in<top (required)="">’ from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in
require' from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:inrequire’ from /usr/lib/ruby/1.9.1/rubygems/specification.rb:7:in
<top (required)="">' from /usr/lib/ruby/1.9.1/rubygems.rb:206:intry_activate’ from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:59:in
rescue in require' from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:35:inrequire’ from /etc/sensu/plugins/community/elasticsearch/check-es-cluster-status.rb:31:in `’ .
Within seconds (probably the next time the check runs), the alert is resolved:
RESOLVED - [elasticsearch02/elasticsearch_cluster_status_check] - ESClusterStatus OK: Cluster is green .
The checks generally work (including properly detecting when something is actually wrong), so I know the right gems are all installed. At the times these alerts happens, nothing special is happening on the boxes (as far as I know).
I do have the boxes configured to automatically apply security updates from Ubuntu; could that be the cause? (ie, at this very moment, some gem was removed while a security update was being applied) That seems far-fetched, and it happens too frequently to be a plausible explanation.
Has anyone else ever seen this? Any ideas?