# Issue with Multi-Tenancy and Tenant-specific Dashboard saving in Opensearch Dashboards

**URL:** <https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098>\
**Category:** Security\
**Created:** [February 27, 2024, 7:36am UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098 "2024-02-27T07:36:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![conan\_kudo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@conan\_kudo](https://forum.opensearch.org/u/conan_kudo)\
**Post date:** [February 27, 2024, 7:36am UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098/1 "2024-02-27T07:36:03Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch Version: 2.11.1  
OpenSearch Dashboards Version: 2.11.1  
Authentication: Keycloak with OpenID Connect

Multi-tenancy is enabled.  
The tenant, ddp role and roles mapping are created in the dashboard ui.  
A custom tenant named `ddp` is created for grouping specific dashboards and visualizations.  
Users authenticated via Keycloak are mapped to corresponding roles in OpenSearch, granting them access to the `ddp` tenant.

**Describe the issue** :

When users try to save a dashboard in the ddp tenant, the operation seems to default to the global tenant, despite the `ddp` tenant being selected.

**Configuration** :

```auto
securityconfig: 
data:
        action_groups.yml: |-
          _meta:
            type: "actiongroups"
            config_version: 2
        internal_users.yml: |-
          _meta:
            type: "internalusers"
            config_version: 2
          admin:
            hash: "{{ .ADMIN_PASSWORD_HASH }}"
            reserved: true
            backend_roles:
            - "admin"
            description: "admin user"
        nodes_dn.yml: |-
          _meta:
            type: "nodesdn"
            config_version: 2
        whitelist.yml: |-
          _meta:
            type: "whitelist"
            config_version: 2
        tenants.yml: |-
          _meta:
            type: "tenants"
            config_version: 2
        roles_mapping.yml: |-
          _meta:
            type: "rolesmapping"
            config_version: 2
          all_access:
            reserved: false
            backend_roles:
            - "admin"
            description: "Maps admin to all_access"
          own_index:
            reserved: false
            users:
            - "*"
            description: "Allow full access to an index named like the username"
          readall:
            reserved: false
            backend_roles:
            - "readall"
          manage_snapshots:
            reserved: false
            backend_roles:
            - "snapshotrestore"
          dashboard_server:
            reserved: true
            users:
            - "dashboarduser"
        roles.yml: |-
          _meta:
            type: "roles"
            config_version: 2
          dashboard_read_only:
            reserved: true
          security_rest_api_access:
            reserved: true
          # Allows users to view monitors, destinations and alerts
          alerting_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/alerting/alerts/get'
              - 'cluster:admin/opendistro/alerting/destination/get'
              - 'cluster:admin/opendistro/alerting/monitor/get'
              - 'cluster:admin/opendistro/alerting/monitor/search'
          # Allows users to view and acknowledge alerts
          alerting_ack_alerts:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/alerting/alerts/*'
          # Allows users to use all alerting functionality
          alerting_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster_monitor'
              - 'cluster:admin/opendistro/alerting/*'
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - 'indices_monitor'
                  - 'indices:admin/aliases/get'
                  - 'indices:admin/mappings/get'
          # Allow users to read Anomaly Detection detectors and results
          anomaly_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/ad/detector/info'
              - 'cluster:admin/opendistro/ad/detector/search'
              - 'cluster:admin/opendistro/ad/detectors/get'
              - 'cluster:admin/opendistro/ad/result/search'
              - 'cluster:admin/opendistro/ad/tasks/search'
              - 'cluster:admin/opendistro/ad/detector/validate'
              - 'cluster:admin/opendistro/ad/result/topAnomalies'
          # Allows users to use all Anomaly Detection functionality
          anomaly_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster_monitor'
              - 'cluster:admin/opendistro/ad/*'
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - 'indices_monitor'
                  - 'indices:admin/aliases/get'
                  - 'indices:admin/mappings/get'
          # Allows users to read Notebooks
          notebooks_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/notebooks/list'
              - 'cluster:admin/opendistro/notebooks/get'
          # Allows users to all Notebooks functionality
          notebooks_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/notebooks/create'
              - 'cluster:admin/opendistro/notebooks/update'
              - 'cluster:admin/opendistro/notebooks/delete'
              - 'cluster:admin/opendistro/notebooks/get'
              - 'cluster:admin/opendistro/notebooks/list'
          # Allows users to read observability objects
          observability_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opensearch/observability/get'
          # Allows users to all Observability functionality
          observability_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opensearch/observability/create'
              - 'cluster:admin/opensearch/observability/update'
              - 'cluster:admin/opensearch/observability/delete'
              - 'cluster:admin/opensearch/observability/get'
          # Allows users to read and download Reports
          reports_instances_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/reports/instance/list'
              - 'cluster:admin/opendistro/reports/instance/get'
              - 'cluster:admin/opendistro/reports/menu/download'
          # Allows users to read and download Reports and Report-definitions
          reports_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/reports/definition/get'
              - 'cluster:admin/opendistro/reports/definition/list'
              - 'cluster:admin/opendistro/reports/instance/list'
              - 'cluster:admin/opendistro/reports/instance/get'
              - 'cluster:admin/opendistro/reports/menu/download'
          # Allows users to all Reports functionality
          reports_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/reports/definition/create'
              - 'cluster:admin/opendistro/reports/definition/update'
              - 'cluster:admin/opendistro/reports/definition/on_demand'
              - 'cluster:admin/opendistro/reports/definition/delete'
              - 'cluster:admin/opendistro/reports/definition/get'
              - 'cluster:admin/opendistro/reports/definition/list'
              - 'cluster:admin/opendistro/reports/instance/list'
              - 'cluster:admin/opendistro/reports/instance/get'
              - 'cluster:admin/opendistro/reports/menu/download'
          # Allows users to use all asynchronous-search functionality
          asynchronous_search_full_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/asynchronous_search/*'
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - 'indices:data/read/search*'
          # Allows users to read stored asynchronous-search results
          asynchronous_search_read_access:
            reserved: true
            cluster_permissions:
              - 'cluster:admin/opendistro/asynchronous_search/get'
          # Allows user to use all index_management actions - ism policies, rollups, transforms
          index_management_full_access:
            reserved: true
            cluster_permissions:
              - "cluster:admin/opendistro/ism/*"
              - "cluster:admin/opendistro/rollup/*"
              - "cluster:admin/opendistro/transform/*"
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - 'indices:admin/opensearch/ism/*'
          # Allows users to use all cross cluster replication functionality at leader cluster
          cross_cluster_replication_leader_full_access:
            reserved: true
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - "indices:admin/plugins/replication/index/setup/validate"
                  - "indices:data/read/plugins/replication/changes"
                  - "indices:data/read/plugins/replication/file_chunk"
          # Allows users to use all cross cluster replication functionality at follower cluster
          cross_cluster_replication_follower_full_access:
            reserved: true
            cluster_permissions:
              - "cluster:admin/plugins/replication/autofollow/update"
            index_permissions:
              - index_patterns:
                  - '*'
                allowed_actions:
                  - "indices:admin/plugins/replication/index/setup/validate"
                  - "indices:data/write/plugins/replication/changes"
                  - "indices:admin/plugins/replication/index/start"
                  - "indices:admin/plugins/replication/index/pause"
                  - "indices:admin/plugins/replication/index/resume"
                  - "indices:admin/plugins/replication/index/stop"
                  - "indices:admin/plugins/replication/index/update"
                  - "indices:admin/plugins/replication/index/status_check"
        config.yml: |-
          _meta:
            type: "config"
            config_version: "2"
          config:
            dynamic:
              http:
                anonymous_auth_enabled: false
              do_not_fail_on_forbidden: true
              do_not_fail_on_forbidden_empty: true  
              authc:
                basic_internal_auth_domain:
                  http_enabled: true
                  transport_enabled: true
                  order: "4"
                  http_authenticator:
                    type: basic
                    challenge: true
                  authentication_backend:
                    type: intern
                openid_auth_domain:
                  http_enabled: true
                  transport_enabled: true
                  order: "0"
                  http_authenticator:
                    type: openid
                    challenge: false
                    config:
                      subject_key: preferred_username
                      roles_key: groups
                      openid_connect_url: "https://id.abc.com/realms/ddp/.well-known/openid-configuration"
                  authentication_backend:
                    type: noop

```

opensearch dashboard and nodepools:

```auto
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
  name: opensearch-cluster
  namespace: logging
spec:
  general:
    httpPort: 9200
    serviceName: opensearch-cluster
    version: 2.11.1
    drainDataNodes: true 
    setVMMaxMapCount: true 
  security:
    tls:  
      transport:  
        generate: true  
        perNode: true  
      http:
        generate: true
    config:  
      securityConfigSecret:
        name: securityconfig-secret 
      adminCredentialsSecret:
        name: admin-credentials-secret  
  dashboards:
    env:
      - name: OPENID_CLIENT_ID
        valueFrom:
          secretKeyRef:
            name: openid-client-secret
            key: OPENID_CLIENT_ID
      - name: OPENID_CLIENT_SECRET
        valueFrom:
          secretKeyRef:
            name: openid-client-secret
            key: OPENID_CLIENT_SECRET
    labels:
      app: Opensearch
    additionalConfig:
      opensearch_security.auth.type: "openid"
      opensearch_security.auth.multiple_auth_enabled: "true"
      opensearch_security.openid.connect_url: "xxx"
      opensearch_security.openid.base_redirect_url: "xxx"
      opensearch_security.openid.client_id: "${OPENID_CLIENT_ID}"
      opensearch_security.openid.client_secret: "${OPENID_CLIENT_SECRET}"
      opensearch_security.multitenancy.enabled: "true"
      opensearch_security.multitenancy.tenants.enable_global: "true"
      opensearch_security.multitenancy.tenants.enable_private: "true"
      opensearch_security.multitenancy.enable_filter: "true"  
    version: 2.11.1
    enable: true
    replicas: 1
    opensearchCredentialsSecret:
      name: admin-credentials-secret 
    tls:
      enable: false
      generate: false
    resources:
      requests:
         memory: "512Mi"
         cpu: "200m"
      limits:
         memory: "2Gi"
         cpu: "500m"

  nodePools:
    - component: nodes
      replicas: 3
      diskSize: "200Gi"
      nodeSelector:     
      resources:
         requests:
            memory: "2Gi"
            cpu: "500m"
         limits:
            memory: "2Gi"
            cpu: "1000m"
      roles:
        - "cluster_manager"
        - "data"
      labels:
        app: Opensearch

```

---

<div class="post-metadata">

**Author:** ![Mantas](https://avatars.discourse-cdn.com/v4/letter/m/7bcc69/32.png) [@Mantas](https://forum.opensearch.org/u/Mantas)\
**Post date:** [February 27, 2024, 11:09am UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098/2 "2024-02-27T11:09:04Z")

</div>

Hi @conan_kudo,

Is this happening to all users or this behavior is specific to a single or a group of users?

Thanks,  
mj

---

<div class="post-metadata">

**Author:** ![conan\_kudo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@conan\_kudo](https://forum.opensearch.org/u/conan_kudo)\
**Post date:** [February 27, 2024, 12:44pm UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098/3 "2024-02-27T12:44:06Z")

</div>

Hey @Mantas ,

thanks a ton for replying. This unfo applies to all users.

In the end, i want different users from different teams to be able to manage their indexes and dashboards, and dashboard objects shouldnt leak across teams.

It seems that the .kibana\_hash\_xxx indexes seem to not get created 😕

With my admin user (user that is assigned to all\_access) i can see that there is only the .kibana\_1 index.

GET /\_cat/indices/.kibana\*

green open .kibana\_1 8J2xSCkuQx682bTf7XrTaQ 1 1 1 0 10.3kb 5.1kb

I must be missing something. Any ideas?

Kind regards

---

<div class="post-metadata">

**Author:** ![Mantas](https://avatars.discourse-cdn.com/v4/letter/m/7bcc69/32.png) [@Mantas](https://forum.opensearch.org/u/Mantas)\
**Post date:** [February 29, 2024, 2:16pm UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098/4 "2024-02-29T14:16:25Z")

</div>

Hi @conan_kudo,

I can’t see your `tenants.yml` (see sample below), would you mind sharing your `tenants.yml`?

sample:

```auto
        tenants.yml: |-
          _meta:
            type: "tenants"
            config_version: 2

```

Thanks,  
mj

---

<div class="post-metadata">

**Author:** ![cwperks](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/cwperks/32/4968_2.png) [@cwperks](https://forum.opensearch.org/u/cwperks)\
**Post date:** [November 27, 2024, 1:49pm UTC](https://forum.opensearch.org/t/issue-with-multi-tenancy-and-tenant-specific-dashboard-saving-in-opensearch-dashboards/18098/5 "2024-11-27T13:49:43Z")

</div>

@conan_kudo Are you using a proxy? When proxying requests from Dashboards → OpenSearch backend make sure all headers are forwarded. The security plugin relies on the headers from [security/src/main/java/org/opensearch/security/auth/BackendRegistry.java at main · opensearch-project/security · GitHub](https://github.com/opensearch-project/security/blob/main/src/main/java/org/opensearch/security/auth/BackendRegistry.java#L464-L466) to determine the tenant a request is destined for.