Having issues with client.json not substituting values and possibly rubygem-json-1.5.5-3

Sensu Classic (EOL)
1

We had a small unexpected outage yesterday but when bringing up the cluster again all nodes except for the stateful nodes (which have hardcoded values in place) we have on hand would fail to start the sensu-client.

Upon digging into /var/log/sensu/sensu-client.log we found the following error.

{“timestamp”:“2014-10-31T13:33:13.561229-0500”,“level”:“fatal”,“message”:“client name cannot contain spaces or special characters”,“object”:{“name”:"HOSTNAME","address":"grep (hostname -s) /etc/hosts | awk ‘{print $1}’",“subscriptions”:[“blues”,“compute”]}}

client.json on each of our stateless nodes has the following.

{
“client”: {
“name”: “hostname", "address": "grep (hostname -s) /etc/hosts | awk ‘{print $1}’ | head -n1”,
“subscriptions”: [ “blues”, “compute” ]
}
}

Before the outage everything worked fine and the values would get substituted as one would expect. During the outage things were updated and as far as I can see the only guilty looking culprit I see is an update to rubygem-json

Updated     rubygem-json-1.5.5-1.el6.x86_64           @updates
Update                              1.5.5-3.el6.x86_64           @updates

If anyone has a different explanation that would very helpful.

Thanks,
JB

2

Update downgrading rubygem-json back to original version from a working setup does not yield a functioning setup. For some reason this has just started acting up.

···

On Friday, October 31, 2014 2:13:10 PM UTC-5, John B wrote:

We had a small unexpected outage yesterday but when bringing up the cluster again all nodes except for the stateful nodes (which have hardcoded values in place) we have on hand would fail to start the sensu-client.

Upon digging into /var/log/sensu/sensu-client.log we found the following error.

{“timestamp”:“2014-10-31T13:33:13.561229-0500”,“level”:“fatal”,“message”:“client name cannot contain spaces or special characters”,“object”:{“name”:“$HOSTNAME”,“address”:“grep $(hostname -s) /etc/hosts | awk ‘{print $1}’”,“subscriptions”:[“blues”,“compute”]}}

client.json on each of our stateless nodes has the following.

{
“client”: {
“name”: “$hostname”,
“address”: “grep $(hostname -s) /etc/hosts | awk ‘{print $1}’ | head -n1”,
“subscriptions”: [ “blues”, “compute” ]
}
}

Before the outage everything worked fine and the values would get substituted as one would expect. During the outage things were updated and as far as I can see the only guilty looking culprit I see is an update to rubygem-json

Updated     rubygem-json-1.5.5-1.el6.x86_64           @updates
Update                              1.5.5-3.el6.x86_64           @updates

If anyone has a different explanation that would very helpful.

Thanks,
JB

3

It would seem that you are attempting to render the Sensu client config, using shell. Sensu has always expected valid JSON, it would have always expected the client config already be populated.

···

On Oct 31, 2014 12:28 PM, “John B” wilshire461@gmail.com wrote:

Update downgrading rubygem-json back to original version from a working setup does not yield a functioning setup. For some reason this has just started acting up.

On Friday, October 31, 2014 2:13:10 PM UTC-5, John B wrote:

We had a small unexpected outage yesterday but when bringing up the cluster again all nodes except for the stateful nodes (which have hardcoded values in place) we have on hand would fail to start the sensu-client.

Upon digging into /var/log/sensu/sensu-client.log we found the following error.

{“timestamp”:“2014-10-31T13:33:13.561229-0500”,“level”:“fatal”,“message”:“client name cannot contain spaces or special characters”,“object”:{“name”:“$HOSTNAME”,“address”:“grep $(hostname -s) /etc/hosts | awk ‘{print $1}’”,“subscriptions”:[“blues”,“compute”]}}

client.json on each of our stateless nodes has the following.

{
“client”: {
“name”: “$hostname”,
“address”: “grep $(hostname -s) /etc/hosts | awk ‘{print $1}’ | head -n1”,
“subscriptions”: [ “blues”, “compute” ]
}
}

Before the outage everything worked fine and the values would get substituted as one would expect. During the outage things were updated and as far as I can see the only guilty looking culprit I see is an update to rubygem-json

Updated     rubygem-json-1.5.5-1.el6.x86_64           @updates
Update                              1.5.5-3.el6.x86_64           @updates

If anyone has a different explanation that would very helpful.

Thanks,
JB

4

I guess so. Upon getting another node up and hardcoding the name but not the ip I got from the API when querying the node’s IP addesss

grep $(hostname -s) /etc/hosts | awk ‘{print $1}’ | head -n1

Still curious why it would work for several months and then decide to quit parsing without any updates…

···

On Friday, October 31, 2014 2:35:36 PM UTC-5, portertech wrote:

It would seem that you are attempting to render the Sensu client config, using shell. Sensu has always expected valid JSON, it would have always expected the client config already be populated.