Sensu web error on launch/setup

Sensu Go
,
1

Hi,

I just upgraded from Sensu Go 5.21 to 6.0 and one of the changes is the need for setting up Sensu Web separately. Here are the steps i took to launch the web module on my AWS EC2 medium instance running Centos 7.6

cd /tmp

curl -s --location https://rpm.nodesource.com/setup_10.x | bash -
yum install -y nodejs
curl -s --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo
yum install -y yarn

wget https://github.com/sensu/web/archive/v1.0.1.tar.gz
tar -xvf v1.0.1.tar.gz
cd web-1.0.1
yarn install
NODE_ENV=production PORT=80 API_URL=https://<MY BACKEND URL>:8080 yarn node scripts serve

When i try to access the UI on the browser, i get timed out and looking at the logs i see the following

yarn node v1.22.4[HPM] Proxy created: [ '/auth', '/graphql', '/api' ]  ->  https://<MY BACKEND URL>:8080
clean-webpack-plugin: /tmp/web-1.0.1/build/app has been removed.listening on 80
(node:12179) DeprecationWarning: Tapable.apply is deprecated. Call apply on the plugin directly instead(node:12179) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
<e> [webpack-dev-middleware] Child "app":<e> ERROR in ./src/lib/buildInfo.macro.js
<e> Module build failed (from ./config/macroLoader.js):<e> Error: [git-rev-sync] no git repository found
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:47:11)<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at branch (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:79:16)<e>     at long (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:92:11)
<e>     at Object.short (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:118:10)<e>     at Module.module.exports.1Icq.__webpack_exports__.default (/tmp/web-1.0.1/src/lib/buildInfo.macro.js:98:34)
<e>     at Object.default (/tmp/web-1.0.1/config/macroLoader.js:9:27)<e>  @ ./src/lib/buildInfo.tsx 6:0-37 8:30-34
<e>  @ ./src/lib/component/util/useBuildInfo.js<e>  @ ./src/lib/component/util/index.tsx
<e>  @ ./src/app/index.js<e>  @ multi ./src/app
<i> [webpack-dev-middleware] Child "app": Failed to compile.Error from chokidar (/tmp/web-1.0.1/node_modules/intl/locale-data/jsonp): Error: ENOSPC: System limit for number of file watchers reached, watch '/tmp/web-1.0.1/node_modules/intl/locale
-data/jsonp/smn-FI.js'Error from chokidar (/tmp/web-1.0.1/node_modules/intl/locale-data/jsonp): Error: ENOSPC: System limit for number of file watchers reached, watch '/tmp/web-1.0.1/node_modules/intl/locale
-data/jsonp/smn.js'

From the logs it seems like the module needs more space and possibly a minimum spec for the machine which i do not see in the repository or docs. Any help on this is appreciated since i would really want to use the Sensu Go 6.0 version but without the UI setup it becomes difficult. Is there any steps that i am missing or are there docs that i can be pointed to?

2
3

Even with increasing the limit to 2048+, the error persists.

yarn node v1.22.4
[HPM] Proxy created: [ '/auth', '/graphql', '/api' ]  ->  https://10.153.1.228:8080
clean-webpack-plugin: /tmp/web-1.0.1/build/app has been removed.
listening on 80
(node:17629) DeprecationWarning: Tapable.apply is deprecated. Call apply on the plugin directly instead
(node:17629) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
<i> [webpack-dev-middleware] wait until bundle finished: /index.html
<e> [webpack-dev-middleware] Child "app":
<e> ERROR in ./src/lib/buildInfo.macro.js
<e> Module build failed (from ./config/macroLoader.js):
<e> Error: [git-rev-sync] no git repository found
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:47:11)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at _getGitDirectory (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:75:10)
<e>     at branch (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:79:16)
<e>     at long (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:92:11)
<e>     at Object.short (/tmp/web-1.0.1/node_modules/git-rev-sync/index.js:118:10)
<e>     at Module.module.exports.1Icq.__webpack_exports__.default (/tmp/web-1.0.1/src/lib/buildInfo.macro.js:98:34)
<e>     at Object.default (/tmp/web-1.0.1/config/macroLoader.js:9:27)
<e>  @ ./src/lib/buildInfo.tsx 6:0-37 8:30-34
<e>  @ ./src/lib/component/util/useBuildInfo.js
<e>  @ ./src/lib/component/util/index.tsx
<e>  @ ./src/app/index.js
<e>  @ multi ./src/app
<i> [webpack-dev-middleware] Child "app": Failed to compile.
Error from chokidar (/tmp/web-1.0.1/node_modules/@material-ui/icons): Error: ENOSPC: System limit for number of file watchers reached, watch '/tmp/web-1.0.1/node_modules/@material-ui/icons/Battery90Outlined.js'
........

Is there a specific number that is needed?

4

I’m not sure of that. However, I was able to run it on a CentOS 7 server with it set to 8192.

5

Unfortunately it is the same error.

6

Yes, I thought it was working but it took quite a while for mine to error out with that same message.

I was, after some more tuning, able to get rid of that error message with the value set to 160000.

7

I got it running with 524288 which is the max possible number allowed. Thank you for your help

8

Also, to get rid of this error, you need to be working from a clone of the git repo, not an un-tar’ed version of the release.

9

That is correct. I did have to make that change to get it working. Is that always going to be the case? Seems odd that i would need to git clone the repo over a untar version of the release. For my backend i use a compiled version of the application without any issues. Any chance that will be the case in the future for Sensu Web?

10

This is a pretty common problem with React framework as it seems that rely on chokidar through watchpack.
Ref: https://developer.aliyun.com/mirror/npm/package/chokidar/

But i don’t think these are fatal errors. Im able to use the 1.0.1 release on a centos 7.6 small EC2 instance with 2000 user watch inode limit and login to the web-ui without issue.

I generate the file watch errors at web-ui service start up, but I don’t think these errors are operational impact, i think it just has to do with some of the node package management yarn is doing behind the scenes to make sure all the node modules are in place. Yarn does a lot of package management actions on top of running the requested nodejs application service.

If you want to prevent the errors, its common practice is to increase the limit to a large number. The chokidar troubleshooting section in the link above suggests a very large number… as per common practice.

Because the watchers appear to be associated with the node module dependencies and not in the application code as housed in the repo… I’m not really sure what the sufficient number is… I don’t think anyone does…because the node dep tree can be well… large. That being said, setting the limit to 20000 appears to have prevented the errors for me for this node application.

11

To be clear I was able to get inode watch errors from the git clone.
So there’s nothing special about the git versus tarball, except maybe the number of files that chokidar is trying to set watches on.

12

I didn’t mean to correlate the git errors with the inode errors.

13

Without the git clone route, i was not able to get it working. Now that i do have the UI up and running, it seems to disregard the users created via sensuctl.

[root@ip-10-153-1-201 bin]# sensuctl version
sensuctl version 6.0.0+ee, enterprise edition, build 98694fcc4ee5552bcdd4f394aa39e4f0466cbaf2, built 2020-08-07T19:07:20Z, built with go1.13.15
[root@ip-10-153-1-201 bin]# sensu-agent version
sensu-agent version 6.0.0+ee, enterprise edition, build 98694fcc4ee5552bcdd4f394aa39e4f0466cbaf2, built 2020-08-07T19:07:18Z, built with go1.13.15
[root@ip-10-153-1-201 bin]# sensu-backend version
sensu-backend version (devel)+ce, community edition, built with go1.13

These are my app versions. These are the ports open on my instances (including port 80 of the web UI which is hosted on backend server instance)

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1176/master
tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      9878/sensu-backend
tcp        0      0 127.0.0.1:2380          0.0.0.0:*               LISTEN      9878/sensu-backend
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1256/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      1176/master
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd
tcp6       0      0 :::80                   :::*                    LISTEN      11861/node
tcp6       0      0 :::8080                 :::*                    LISTEN      9878/sensu-backend
tcp6       0      0 :::8081                 :::*                    LISTEN      9878/sensu-backend
tcp6       0      0 :::22                   :::*                    LISTEN      1256/sshd

The web UI is extremely slow (as previously noted by Todd) and trying to login with credentials which work for sensuctl seems not work with the UI

t: 500
    at http://<MY BACKEND URL>/static/js/app_0f63.js:2:773725
    at f (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:566485)
    at T._settlePromiseFromHandler (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:555000)
    at T._settlePromise (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:555800)
    at T._settlePromise0 (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:556499)
    at T._settlePromises (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:557826)
    at u._drainQueue (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:520188)
    at u._drainQueues (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:520249)
    at drainQueues (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:518460)
    at MutationObserver.<anonymous> (http://<MY BACKEND URL>/static/js/vendors~app_999e.js:2:563422)

Can i know either of your setup which has the Sensu Web working?

14

My initial test was against a sensu-backend installed from the official packages.

Building sensu-backend now from the open source repository to retest.

15

@swapzero

So i just built a fresh sensu-backend from the sensu-go repository using the v6.0.0 tag. I’m able to login via the web-ui just fine.

Got it up and running, added a user named test_user with sensuctl
and I’m able to login as test_user using the web_ui no problem at all.

My only thought is you have TLS secured backend api enabled (https::/whatever:8080) and are using self signed certs using a private CA that the web_ui isn’t aware of and you’ve either disabled tls verification in your sensuctl or have given your sensuctl the optional private CA file to use to verify the self-signed keys.

I’ve tested my local setup using a secured backend api using certs from Let’s Encrypt, so the root CA is trusted by the operating system so my web-ui has no problems verifying the api endpoint. I’ve also tested a setup with no TLS security on the api endpoint in place just for completeness.

Other than a TSL issue, I’m not why you are having a problem and I am not.

16

I was incorrect in my assumption. I didnot have TLS enabled which meant the issue persisted. I just set the API_URL to http instead of https and it worked fine. Thank you @jspaleta and @todd for your assistance. I do hope the documentation will be updated separately for sensu web on the site in future. Looking forward to the new features