No permissions for [] and User [name=admin, backend_roles=[admin], requestedTenant=null

OpenSearch Dashboards
,
1

Sorry if this is super-straightforward. My Search-Fu and AI skills seem to be coming up short on this one. Would post more images of areas affected by the behavior, but I’m a new user :frowning:

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

  • OpenSearch 3.6.0 (Bare Metal on VM)
  • Dashboards 3.6.0 (Bare Metal on VM)
  • Ubuntu Server 24.04 LTS
  • Firefox 150.0, Win11

Describe the issue:

Receiving security exceptions “No permissions for and User [name=admin, backend_roles=[admin], requestedTenant=null” for select (but not all) areas of Opensearch Dashboards.

Configuration (in part):

opensearch-security/roles.yml:
------------------------------
# No config for all_access...


opensearch-dashboards.yml:
--------------------------
...
opensearch:
   hosts: [https://<opensearch-management-node>:9200]
   username: admin
   password: <redacted>
   requestHeadersAllowlist: [authorization, securitytenant]
   ssl:
      certificate: <admin client cert>
      key: <admin client cert key>
      certificateAuthorities: [ "<path to ca cert>" ]
      verificationMode: none
...
opensearchDashboards:
...
   dashboardAdmin:
      groups: ["admin", "dashboard_admin"]
      users: ["admin"]
...
opensearch_security:
   multitenancy:
      enabled: false
      tenants:
         preferred: [Private, Global]
   readonly_mode:
      roles: [kibana_read_only]
...

opensearch.yml:
---------------
...
plugins:
   security:
      ssl:
         transport:
             keystore_type: PKCS12
             keystore_filepath: <relative path to *.p12 file>
             keystore_alias: 1
             truststore_type: PKCS12
             truststore_filepath: <relative path to *.p12 ca store>
... (http config same)
     allow_unsafe_democertificates: false
     allow_default_init_securityindex: true
     authcz.admin_dn:
        "<DN of admin client certificate>"
        ... (other admin user DNs)
     ...
     restapi:
        roles_enabled: [all_access, security_rest_api_access]
     system_indices:
        enabled: true
        permission:
           enabled: true
### End of Config ###

Relevant Logs or Screenshots:

opensearch-dashboards log:
--------------------------
...
...(200 for url /api/workspaces/_list)
{"type":"log"..."tags":["error","opensearch","data"]..."message":"[security_exception]: no permissions for [] and User [name=admin, backend_roles=[admin], requestedTenant=null]"}
...(200 for url /api/core/capabilities)



<cluster_name>.log (opensearch management node side):
-----------------------------------------------------
...
...[o.o.s.a.BackendRegistry  ]  [<masternode>-1]  No indices:data/read/search permission for user roles [own_index, all_access] to System Indices .opensearch_dashboards_3
...

Workspace Creation:

image
image588×161 17.2 KB

2

@codehacker84 Did you disable multi-tenancy in OpenSearch security plugin (config.yml)?

3

Well…not originally…but when I did, and pushed just that file with securityadmin.sh, it didn’t seem to help. Silly question: do I have to do that on every node (5 node cluster: 3 data, 1 ingest, 1 master), or just one? (EDIT: I’ve tried to keep static configs consistent across the cluster, but I can’t 100% guarantee I didn’t mess it up, somehow).

4

Hey, guys, it’s me, again: I figured that I’d probably corrupted something, so I went ahead and rebuilt the cluster from scratch (assuming that the security indexes were all toast). Unfortunately, I’m getting the same issue, even on a fresh rebuild of 3.6.0. (EDIT: I now have everything snapshotted to right before I start the cluster, so I can rollback to a clean snapshot and try again, fresh, if needed.)

I don’t understand how the admin can be locked out of the system, so I’ve got to assume there’s some option I’m missing in the configs. Any theories what I could be doing wrong? Happy to provide logs or other data.

Is it because I put Dashboards on a separate VM from the rest of the cluster, maybe?

5

Alright. With some fiddling, I figured out that this is related to “plugins.security.system_idices.enable: true” (would seem like a good thing to have, but fouls things up on my deployment, for some reason).

If I turn the setting off, everything works correctly.

If the setting is on, it seems the system ignores my admin certificate and treats “admin” as a regular user.

If I further set “plugins.security.system_indices.permission.enable: true”, it appears that the “all_access” role does not include access to the System Indices, by default. Gonna see if I can just manually add it.

I want to point out that I have no idea why it is this way, it’s just the way it is, on my deployment.

(EDIT: Marking this as A possible solution).

6

@codehacker84 When you have plugins.security.system_idices.enable enabled, you also must list system indices.
Your shared config is missing the system indices list.

plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: ['.opendistro-alerting-config', '.opendistro-ism-*', '.opendistro-reports-*', '.opensearch-notifications-*', '.opensearch-notebooks', '.opensearch-observability', '.opendistro-asynchronous-search-response*', '.replication-metadata-store']