Built new cluster, still says to run securityadmin.sh after it has been run

Security
1

I just built out a new cluster with slightly customized docker images (only doing customized so I can inject the security config files directly into the container), and no matter what I do, the cluster keeps saying that security is not initialized, even after the securityadmin.sh runs and reports no errors.

Ex logs (sensitive info redacted):
Open Distro Security Admin v7
Will connect to localhost:9300 … done
Connected as CN=REDACTED
Elasticsearch Version: 7.9.1
Open Distro Security Version: 1.11.0.0
Contacting elasticsearch cluster ‘elasticsearch’ …
Clustername: REDACTED
Clusterstate: GREEN
Number of nodes: 3
Number of data nodes: 3
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/
Will update ‘_doc/config’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml
SUCC: Configuration for ‘config’ created or updated
Will update ‘_doc/roles’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles.yml
SUCC: Configuration for ‘roles’ created or updated
Will update ‘_doc/rolesmapping’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles_mapping.yml
SUCC: Configuration for ‘rolesmapping’ created or updated
Will update ‘_doc/internalusers’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/internal_users.yml
SUCC: Configuration for ‘internalusers’ created or updated
Will update ‘_doc/actiongroups’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/action_groups.yml
SUCC: Configuration for ‘actiongroups’ created or updated
Will update ‘_doc/tenants’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/tenants.yml
SUCC: Configuration for ‘tenants’ created or updated
Will update ‘_doc/nodesdn’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/nodes_dn.yml
SUCC: Configuration for ‘nodesdn’ created or updated
Will update ‘_doc/whitelist’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/whitelist.yml
SUCC: Configuration for ‘whitelist’ created or updated
Will update ‘_doc/audit’ with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/audit.yml
SUCC: Configuration for ‘audit’ created or updated
Done with success

Then when I try to interact with the cluster:
[2020-12-09T20:34:51,623][ERROR][c.a.o.s.a.BackendRegistry] [REDACTEDNODENAME] Not yet initialized (you may need to run securityadmin)

I’m injecting the following security files:
action_groups.yml
audit.yml
config.yml
internal_users.yml
nodes_dn.yml
roles_mapping.yml
roles.yml
tenants.yml
whitelist.yml

All of these lint out without issue, so I don’t think it’s a YAML syntax issue.

Any help would be appreciated.

2

Not much to go on by here. How many nodes should you have in your cluster? You would get this error if there would be firewalls in between the nodes. What does the cluster health say the same thing for every node?

3

There are 3 nodes. I can netcat between all 3 on port 9200 and 9300. Can’t get cluster health information as running any commands against the cluster produce the above mentioned error about security not being initialized. I do see in the output when the securityadmin script is run that that reports all 3 nodes and shows the cluster health as green.

I’ve destroyed and re-created the cluster more than once to ensure there’s no stale data causing problems.

4

@tfmm I had a very similar issue, at the end it turned out I was using later version of config file in the earlier version of ODFE, surprisingly no errors were thrown which didn’t speed things up.
Maybe try and run same version 1.11.0 ODFE locally and using demo install to generate all the config files, then comparing formats to the ones inside the docker image you are testing. In my case config.yml was the culprit.

1 Like