# What if a script having a command in check took more time than interval?

**URL:** <https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661>\
**Category:** Sensu Go\
**Created:** [March 9, 2020, 1:44pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661 "2020-03-09T13:44:26Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![masifpak](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@masifpak](https://discourse.sensu.io/u/masifpak)\
**Post date:** [March 9, 2020, 1:44pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/1 "2020-03-09T13:44:26Z")

</div>

Hi Geeks,

I have a situation. There is a script with some logic and one command communicate with DB. This command may take more time like 5 minutes or more etc. I am running this bash script in sensu check like. I want to run this check every minute no matter that script took more time. Just give 5 minutes (5 turns one minute each ) and after 5 minutes kill the process and start script again.

> /opt/sensu-plugins-ruby/embedded/bin/CronJobsScheduleMonitor.sh

@todd @jspaleta Please guide me sir accordingly.

---

<div class="post-metadata">

**Author:** ![todd](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/todd/32/936_2.png) [@todd](https://discourse.sensu.io/u/todd)\
**Post date:** [March 9, 2020, 3:33pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/2 "2020-03-09T15:33:51Z")

</div>

Am I understanding this correctly that you essentially would expect to possibly have 5 iterations of the same check running at a time with an interval of 60 seconds and a timeout of 300?

00:00 Check 1 starts run  
00:01 Check 2 starts run  
00:02 Check 3 starts run  
00:03 Check 4 starts run  
00:04 Check 5 starts run  
00:05 Check 1is killed, Check 6 starts run  
00:06 Check 2 is killed, Check 7 starts run

The above scenario will never work with Sensu as it will only spawn one version of the check at any one time. Checks 2-5 would never start, check 1 would timeout at the 5 minute mark and then Check 2 would be started one minute later.

00:00 Check 1 starts run  
00:01 Check 1 still running  
00:02 Check 1 still running  
00:03 Check 1 still running  
00:04 Check 1 still running  
00:05 Check 1 is killed  
00:06 Check 2 starts run  
00:07 Check 2 is still running

---

<div class="post-metadata">

**Author:** ![masifpak](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@masifpak](https://discourse.sensu.io/u/masifpak)\
**Post date:** [March 9, 2020, 4:57pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/3 "2020-03-09T16:57:43Z")

</div>

We have a php script and added a sleep of 10 minutes . We added a time check function which check difference of modification time of pid file and current time. If time difference is 5 or more then script will kill process and reemove pid file. But I call the script using sensu check it wait for to complete sleep time of 10 minutes and then run again. I was expecting that sensu check run the script once a minute and dont care about the status of command. Just pick script from start every minutes and follow the logic in it.

> #!/bin/bash  
> #set -o xtrace  
> #set -x  
> #trap read debug
> 
> DOMAIN\_NAME=[app14.abc.com](http://app14.abc.com)  
> DB\_VERSION=2019080901  
> CRON\_PATH=/var/www/$DOMAIN\_NAME/scrybe/amfphp1\_9/services/db\_services\_$DB\_VERSION/cronjobs  
> JOB\_NAME=CronJobsScheduleMonitor  
> #PID\_PATH=/opt/convo/CronJobsScheduleMonitor/$JOB\_NAME.pid  
> PID\_PATH=/opt/convo/CronJobsScheduleMonitor  
> PID\_FILE=$PID\_PATH/$JOB\_NAME.pid  
> echo “$PID\_PATH”  
> echo “$PID\_FILE”  
> date \>\> /tmp/time.txt  
> PID\_FILE\_EXISTS=“”  
> PROCESS\_RUNNING=“”
> 
> diff\_time()  
> {  
> FTIME=$(date -r CronJobsScheduleMonitor.pid “+%M”)  
> CTIME=$(date “+%M”)  
> echo “$FTIME”  
> echo “$CTIME”  
> ETIME=$( expr $CTIME - $FTIME )  
> echo $ETIME  
> }
> 
> start\_job(){
> 
> ```
> #sudo runuser -l apache -s /bin/bash -c "php72 $CRON_PATH/monitoring/CronJobsScheduleMonitor.php" &
> sudo runuser -l apache -s /bin/bash -c "php72 $CRON_PATH/monitoring/CronJobsScheduleMonitor.php /tmp/logly.log 2>&1 &"
> #sudo runuser -l apache -s /bin/bash -c "php72 $CRON_PATH/monitoring/CronJobsScheduleMonitor.php & "
> PID=$!
> #PID=$(ps -aux | grep CronJobsScheduleMonitor | grep ^apache | awk '{print $2}')
> echo "Started with PID: $PID"
> echo "start new:" $(date) >> /tmp/time.txt
> echo $PID > "$PID_FILE"
> sleep 5s
> CURRENT_PID2=$(ps -aux | grep CronJobsScheduleMonitor | grep ^apache | awk '{print $2}')
> if [-z "$CURRENT_PID2"];
> then
> rm "$PID_FILE"
> echo "OK: All Cronjobs are Working Fine."
> exit 0
> else
> echo "Cronjobs are still running. Please check cronjobs. "
> exit 1
> fi
> 
> ```
> 
> }
> 
> # Check if PID file exists
> 
> if [-f “$PID\_FILE”]; then  
> PID\_FILE\_EXISTS=“YES”  
> else  
> PID\_FILE\_EXISTS=“NO”  
> fi
> 
> # Check if Proceess exists
> 
> PROCECSS\_ID=$(ps -aux | grep CronJobsScheduleMonitor | grep ^apache | awk ’ {print $2} ')  
> if [-z “$PROCECSS\_ID”]; then
> 
> PROCESS\_RUNNING=“NO”  
> else  
> PROCESS\_RUNNING=“YES”  
> fi
> 
> echo “$PROCESS\_PID”
> 
> if [[$PID\_FILE\_EXISTS == “YES” && $PROCESS\_RUNNING == “NO”]];then  
> echo “PID\_FILE exist but Process was not Running.” \>\> /tmp/time.txt  
> rm “$PID\_FILE”  
> #nohup php -c /etc/php.ini “$worker\_name”.php $queue\_name 1\>\> “/opt/scripts/appworker/logs/${worker\_name}\_console.log” 2\>\> “/opt/scripts/appworker/logs/${worker\_name}\_error.log” \< /dev/null &  
> start\_job  
> fi
> 
> if [[$PID\_FILE\_EXISTS == “YES” && $PROCESS\_RUNNING == “YES”]];then  
> echo “PID\_FILE and PROCESS\_ID both exist, Do calculate Grace Time.” \>\> /tmp/time.txt  
> GRACE\_TIME=$(diff\_time)  
> echo “Grace Time: $GRACE\_TIME”  
> #GRACE\_TIME=3  
> if [“$GRACE\_TIME” -gt 4];  
> then  
> #PID1=$(ps -aux | grep CronJobsScheduleMonitor | grep ^apache | awk ’ {print $2} ')  
> echo “Process ID is: $PROCESS\_ID”  
> kill -9 $PROCESS\_ID  
> rm “$PID\_FILE”  
> echo “Run After Killing PID”  
> start\_job  
> else  
> echo "Cronjobs are still running. Please check cronjobs. "  
> exit 1  
> fi
> 
> fi
> 
> if [[$PID\_FILE\_EXISTS == “NO” && $PROCESS\_RUNNING == “NO”]];then  
> echo “Both PID\_FILE and PROCESS\_ID do not exist.” \>\> /tmp/time.txt  
> start\_job  
> fi
> 
> if [[$PID\_FILE\_EXISTS == “NO” && $PROCESS\_RUNNING == “YES”]];then  
> echo "PID\_FILE does not exist but Process is Running. \>\> /tmp/time.txt  
> #PID2=$(ps -aux | grep CronJobsScheduleMonitor | grep ^apache | awk ’ {print $2} ')  
> echo “Process ID is: $PROCEESS\_ID”  
> kill -9 $PROCESS\_ID  
> start\_job  
> fi

---

<div class="post-metadata">

**Author:** ![todd](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/todd/32/936_2.png) [@todd](https://discourse.sensu.io/u/todd)\
**Post date:** [March 9, 2020, 5:22pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/4 "2020-03-09T17:22:32Z")

</div>

My question now becomes, if you don’t care about the status of the command, then why have Sensu run it as a check? You can run a check in Sensu for one or two reasons, to do a service check (you care about the exit status of the command) and/or for metrics collection.

From the sounds of this, you’d be just as well to let cron schedule and run the job for you, unless I’m missing some other reason for having Sensu run it.

---

<div class="post-metadata">

**Author:** ![masifpak](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@masifpak](https://discourse.sensu.io/u/masifpak)\
**Post date:** [March 9, 2020, 5:37pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/5 "2020-03-09T17:37:52Z")

</div>

Actually I just want to ensure that no matter status or output of command, ideally it should run smoothly every minute and if there is some problem them must start forcefully after 5 minutes.

---

<div class="post-metadata">

**Author:** ![todd](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/todd/32/936_2.png) [@todd](https://discourse.sensu.io/u/todd)\
**Post date:** [March 9, 2020, 6:10pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/6 "2020-03-09T18:10:44Z")

</div>

Then set the check’s timeout value to 300 (5 minutes). It will kill a check that has taken over five minutes. And will start again at the next interval.

---

<div class="post-metadata">

**Author:** ![masifpak](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@masifpak](https://discourse.sensu.io/u/masifpak)\
**Post date:** [March 9, 2020, 6:14pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/7 "2020-03-09T18:14:29Z")

</div>

After timeout 300 s, if sensu kill the check, how can we say that script or command running by this check will also be killed and start new.

---

<div class="post-metadata">

**Author:** ![todd](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.sensu.io/todd/32/936_2.png) [@todd](https://discourse.sensu.io/u/todd)\
**Post date:** [March 9, 2020, 7:35pm UTC](https://discourse.sensu.io/t/what-if-a-script-having-a-command-in-check-took-more-time-than-interval/1661/8 "2020-03-09T19:35:54Z")

</div>

Any processes spawned by the check _should_ be terminated when the check process is terminated due to timeout.