Looking for some additional info regarding datastore and where are the sensugo users saved.
We have successfully changed datastore to postgres, and all events and filters are being stored there.
But not the users that we created. We have a small team of people that each have their own account, and all these accounts are stored in etcd (confirmed by stopping sensugo, moving etcd directory, letting sesnugo recreate it - no one was able to login, except with the default login which we disabled in our installation). on restoring sensugo etcd, the logins were working fine normally.
We additionally did not find any reference to users in our sensugo database.
The documentation is lacking any information on this - can the users be moved to postgres datastore? what is the procedure for such a move?
Currently, when PostgreSQL is enabled, it is used solely for event storage. All other configuration data continues to be stored in etcd. That is why in the docs that all references to PostgreSQL specifically refer to event storage.
When configured with a PostgreSQL event store, Sensu connects to PostgreSQL to store and retrieve event data in place of etcd. Etcd continues to store Sensu entity and configuration data. You can access event data stored in PostgreSQL using the same Sensu web UI, API, and sensuctl processes as etcd-stored events.
so it is just a event datastore and etcd is for everything else?
Is there, perhaps, a plan to have everything moved to postgres?
And if I may ask, why arent the users moved to postgres? In the current environment of dockers, disposable nodes and etc, seems like moving all this critical data to some kind of a database (sql or nosql) with separate replication, backups and etc seems like a logical step. Just curious.
Hey,
I’m a little confused… etcd is a database… its just not a sql database.
Maybe your just not familiar with etcd itself and your experience is limited to the embedded etcd that sensu provides to make it easy to get started.
Etcd can be run externally as a separate service cluster, and in production scale deployments that’s preferable to relying on the emdedded etcd service sensu provides. The design of etcd includes replication and snapshotting facilities, so I’m a little confused as to your statement. Maybe you mean you want somethng like RDS for etcd as a managed service from your cloud provider that does all the replication and backup configuration for you with sane defaults?
etcd is designed for infrequently changing state in a distributed environment and it works great as a high availability service when run externally. sensu user definitions are exactly the sort of things etcd works great for in distributed environment. Etcd even has a concept of replicators so you can stream changes from one cluster to another…its how sensu commercial federation feature works to get a global control with separate clusters in different regions.
Sensu events however are different than the rest of the config because there is much higher volume of writes and PostgreSQL is really optimized for high event throughput at scale, but it takes more work to tune, and if your using something like rds instead of your own PostgreSQL, you still might not have the control necessary to optimize the PostgreSQL tuning… managed PostgresSQL has trade-offs.
That being said, for a get up and using Sensu for the first time the embedded etcd for everything really drops the learning curve and the configuration requirements. But for production you should really move away from the embedded etcd and use an external etcd… once you stand up an external etcd (and optionally postgres for events) its much easier to autoscale the sensu-backend instances (operating as etcd clients with no embedded etcd) you need to better match the event throughput.