Thank you again for your advice 
We are using these sites as references:
The first link seems to be outdated and we’re not particularly sure on the configuration of Sensu to Logstash.
We also intentionally stopped the beats service since we want to use Sensu as our metrics collection engine.
We updated our configurations but we still can’t get the Sensu checks to ELK stack:
# /etc/sensu/conf.d/check.json
{“checks”: {
"CHECK": {
"type": "metric",
"command": "check-ports.rb -h 192.168.30.187 -p 3000,15672 -t 30",
"subscribers": ["ALL"],
"interval": 60,
"handlers": ["logstash"]
}
}
}
# /etc/sensu/conf.d/handler.json
{
“handlers”: {
"logstash": {
"type": "tcp",
"socket": {
"host": "localhost",
"port": 5514
}
}
}
}
# /etc/logstash/conf.d/logstash.conf
input {
tcp {port => 5514
codec => json
}
}
filter {
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{[@metadata][sensu]}-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
···
On Mon, Mar 13, 2017 at 10:54 PM, Kyle Anderson kyle@xkyle.com wrote:
But the thing that listens for tcp must be running (logstash/beats). You must get that up and running fully first, otherwise sensu won’t be able to talk to it over tcp.
It sounds like you may have mixed the configuration for a command (pipe) handler and tcp handlers. With TCP handlers sensu is just going to send events to something else via tcp, it is not going to run ‘/usr/local/bin/handler-logstash.rb’ so the ‘command’ option in the handler configuration is not correct. What documentation are you looking at? I believe the ‘logstash.conf’ and ‘command’ in the tcp handler are misleading in your configuration and don’t represent what you are trying to do. (probably represent your first attempt using a pipe handler)
On Mon, Mar 13, 2017 at 12:21 AM, Keeshia San Pascual sanpascual.keeshia@gmail.com wrote:
Thank you for your response! we modified the check to “metric” but still, we cannot see the check results(Sensu) to kibana and the logs( logstash-plain.log) only displays info about filebeat (service is not running):
[2017-03-07T09:38:33,368][INFO ][logstash.pipeline ] Starting pipeline {“id”=>“main”, “pipeline.workers”=>2, “pipeline.batch.size”=>125, “pipeline.batch.delay”=>5, “pipeline.max_inflight”=>250}
[2017-03-07T09:38:33,809][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>“0.0.0.0:5044”}
[2017-03-07T09:38:33,842][INFO ][logstash.pipeline ] Pipeline main started
[2017-03-07T09:38:33,904][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2017-03-07T10:42:51,794][WARN ][logstash.runner ] SIGTERM received. Shutting down the agent.
[2017-03-07T10:42:51,801][WARN ][logstash.agent ] stopping pipeline {:id=>“main”}
[2017-03-07T10:42:57,591][WARN ][logstash.shutdownwatcher ] {“inflight_count”=>0, “stalling_thread_info”=>{“other”=>[{“thread_id”=>23, “name”=>“[main]<beats”, “current_call”=>“[…]/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.12-java/lib/logstash/inputs/beats.rb:213:in `run’”}, {“thread_id”=>20, “name”=>“[main]>worker0”, “current_call”=>“[…]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:127:in `synchronize’”}, {“thread_id”=>21, “name”=>“[main]>worker1”, “current_call”=>“[…]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:127:in `synchronize’”}]}}
[2017-03-07T10:42:57,593][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
floadmin@flosensu1:/var/log/logstash$
so we’re not sure how to verify if the Sensu checks are sending to logstash and if our configurations are working. Kindly advise.
Thanks again.
On Sat, Mar 11, 2017 at 1:03 PM, Kyle Anderson kyle@xkyle.com wrote:
If you want a) (all events sent to logstash) then the checks need to be “metric” checks
https://sensuapp.org/docs/latest/reference/checks.html#check-attributes
I agree that using a handler like this is expensive cpu-wise.
On Thu, Mar 9, 2017 at 3:06 AM, Keeshia San Pascual sanpascual.keeshia@gmail.com wrote:
Really appreciate your response! we actually wanted to monitor everything so that would be a). Can you advise what’s missing on our config? and would it be okay if you can run a sanity check? we are blocked and we really can’t figure out what’s wrong.
Thank you again.
On Thu, Mar 9, 2017 at 5:22 PM, Alexander Skiba ghostlyrics@gmail.com wrote:
What do you expect to happen?
a) every check result is written to logstash, regardless of exit code (aka. OK, Warn, etc) OR
b) every negative result is written to logstash (Warning, Critical, Unknown, but not OK)
from a cursory view of the snippets provided by you, that would be b).
Note that I have not sanity checked your config.
On Wednesday, March 8, 2017 at 1:41:43 PM UTC+1, Kaye wrote:
We are trying to send Sensu events to Logstash using tcp handler and we installed sensu-plugins-logstash to do the job but it doesn’t seem to be working and we are currently blocked in solving this as there are no significant logs that can be seen in both sensu-server and in logstash to identify if there are problems in sending, retrieval, or if there are issues in the configurations.