Role mappings not working when using OIDC

Hi (pablo?),

  • OpenSearch: v2.1.0
  • Authentication via KeyCloak (OpenID) v11.0.2

Following up on Backend_roles in roles mapping seems to be ignored when using OpenID
I have the issue that no other role than all_access is being interpreted by opensearch.
underneath all my configs which I apply on all my nodes by running the securityAdmin.sh script.

config.yml

_meta:
          type: "config"
          config_version: 2
        config:
          dynamic:
            authc:
              basic_internal_auth_domain:
                http_enabled: true
                transport_enabled: true
                order: 0
                http_authenticator:
                  type: basic
                  challenge: false
                authentication_backend:
                  type: intern

              openid_auth_domain:
                http_enabled: true
                transport_enabled: true
                order: 1
                http_authenticator:
                  type: openid
                  challenge: false
                  config:
                    openid_connect_idp:
                      enable_ssl: false
                    subject_key: preferred_username
                    roles_key: roles
                    openid_connect_url: "https://auth.{{ opensearch_dashboard_oidc_subdomain }}.{{ common_name }}/auth/realms/{{ opensearch_dashboard_oidc_realm }}/.well-known/openid-configuration"
                    verify_hostnames: false
                authentication_backend:
                  type: noop

internal_users.yml

# This is the internal user database
        # The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh

        _meta:
          type: "internalusers"
          config_version: 2

        # Define your internal users here

        ## Demo users

        {{ op_config.username }}:
          hash: "{{ opensearch.passwords.op | string | password_hash('bcrypt') }}"
          reserved: true
          backend_roles:
          - "admin"
          description: "DSH opensearch admin user"

        anomalyadmin:
          hash: "{{ opensearch.passwords.op | string | password_hash('bcrypt') }}"
          reserved: false
          opendistro_security_roles:
          - "anomaly_full_access"
          description: "Demo anomaly admin user, using internal role"

        kibanaserver:
          hash: "{{ opensearch.passwords.op_dashboard | string | password_hash('bcrypt') }}"
          reserved: true
          description: "Demo OpenSearch Dashboards user"

        kibanaro:
          hash: "{{ opensearch.passwords.op_dashboard_ui | string | password_hash('bcrypt') }}"
          reserved: false
          backend_roles:
          - "kibanauser"
          - "readall"
          attributes:
            attribute1: "value1"
            attribute2: "value2"
            attribute3: "value3"
          description: "Demo OpenSearch Dashboards read only user, using external role mapping"

        logstash:
          hash: "{{ opensearch.passwords.logstash | string | password_hash('bcrypt') }}"
          reserved: false
          backend_roles:
          - "logstash"
          description: "Demo logstash user, using external role mapping"

        readall:
          hash: "{{ opensearch.passwords.op | string | password_hash('bcrypt') }}"
          reserved: false
          backend_roles:
          - "readall"
          description: "Demo readall user, using external role mapping"

        snapshotrestore:
          hash: "{{ opensearch.passwords.op | string | password_hash('bcrypt') }}"
          reserved: false
          backend_roles:
          - "snapshotrestore"
          description: "Demo snapshotrestore user, using external role mapping"

roles_mapping.yml

_meta:
          type: "rolesmapping"
          config_version: 2
        all_access:
          reserved: false
          backend_roles:
            - "admin"
          users:
{% for user in opensearch.admin_users %}
            - {{ user }}
{% endfor %}
          description: "Maps admin to all_access"
        readall:
          reserved: false
          hidden: false
          backend_roles:
          - "readall"
          hosts: []
          users: []
          and_backend_roles: []
        logstash:
          reserved: false
          hidden: false
          backend_roles:
          - "logstash"
          hosts: []
          users:
            - logstash
          and_backend_roles: []
        kibana_server:
          reserved: true
          hidden: false
          backend_roles: []
          hosts: []
          users:
          - "kibanaserver"
          and_backend_roles: []
        own_index:
          reserved: false
          hidden: false
          backend_roles: []
          hosts: []
          users:
          - "*"
          and_backend_roles: []
          description: "Allow full access to an index named like the username"
        kibana_user:
          reserved: false
          hidden: false
          backend_roles:
          - "kibanauser"
          hosts: []
          users: []
          and_backend_roles: []
          description: "Maps kibanauser to kibana_user"






I guess the kc realm roles propagate to the backend-roles in opensearch (see testrole as a proof)

Now 2 things are an issue here:

  • If I add me as a user in roles_mapping.yml in e.g. kibana_user or read_all, I rerun securityAdmin.sh, I can NOT see any indices or menu (left pane).
    On the other hand, If I were to remove the whole openid connect AND roles_mapping.yml file and I would assign me these same roles (since these are already roles defined in opensearch-dashboard), or I login with user kibanaro for example, I CAN see indices and menu.
    This story is different if I put my username in the all_access role in roles_mapping.yml . then I DO get to see indices and menu
  • If I remove roles_mapping.yml and I assign me these same roles as above through keycloak (as you can see the mapping works through my mapper) I can NOT see indices nor menu.

Issue1: roles_mapping.yml file does not work properly
Issue2: keycloak mapping does not work.

As an example: adding me in the kibana_user role, not seeing anything anymore:



@Raki I’ve just tested your settings and it seems that your Role Mapper is incorrectly set.

You don’t have Multivalued enabled. Roles are sent as an array in JWT token.
i.e.
image

Multivalued must be enabled even when you send only single role.
That’s why you have square brackets in the roles view in OpenSearch Dashboards UI.

image

Also, you should use kibanauser instead of kibana_user for the built-in backend role.

Hi @pablo ,
Seems that when I apply my kc config I had set multivalued already to ‘true’. I guess there is something wrong in the kc api, because it is not picked up.
op4

Now I set it to true and as expected I still don’t see any data:



Seems also the realm role is the only one that is applied as the backend-role.
The client role is not propagated into opensearch-dashboard at all.
With the above settings I have added myself as a user within the role kibana_user in roles_mapping.yml.
When I remove myself from that list I see the exact same settings as above, and also no data.

@Raki Now, with the “Multivalued” set to true, you have backed role kibanauser assigned to kibana_user role. Before, any authenticated user had the all_access role assigned.

Could you share your opensearch_dashboards.yml file?

opensearch_dashboards.yml

opensearch_dashboards.yml: |
    server:
      name: dashboards
      host: "0.0.0.0"

    opensearch.hosts: ["https://opensearch-cluster-master.monitoring.svc:9200"]
    opensearch.username: "{{ op_config.username }}"
    opensearch.password: "{{ opensearch.passwords.op | string }}"

    opensearch.ssl.verificationMode: none
    server.ssl.enabled: false

    opensearch_security.auth.type: "openid"
    opensearch_security.openid.connect_url: "https://auth.{{ opensearch_dashboard_oidc_subdomain }}.{{ common_name }}/auth/realms/{{ opensearch_dashboard_oidc_realm }}/.well-known/openid-configuration"
    opensearch_security.openid.client_id: "opensearch-dashboard-sso"
    opensearch_security.openid.client_secret: "{{ keycloak_client_secrets | dict_lookup('opensearch-dashboard-sso', error=True)}}"
    opensearch_security.openid.verify_hostnames: "false"
    opensearch_security.openid.base_redirect_url: "https://opensearch-dashboard.monitoring.{{ common_name }}"
    
    opensearch.requestHeadersWhitelist: ["securitytenant", "Authorization"]
    opensearch_security.multitenancy.enabled: true
    opensearch_security.multitenancy.tenants.enable_global: true
    opensearch_security.multitenancy.tenants.enable_private: true
    opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
    opensearch_security.multitenancy.enable_filter: false

    # kibana_all_read kibanauser
    opensearch_security.readonly_mode.roles: [ "kibana_user", "readall" ]

@Raki As I’ve mentioned in my last comment, you might have the read-only mode assigned to kibana_user role.

In your config, you’ve assigned kibana_user and readll roles.

  # kibana_all_read kibanauser
    opensearch_security.readonly_mode.roles: [ "kibana_user", "readall" ]

The opensearch_security.readonly_mode.roles disables all items in the OpenSearch Dasbhoards UI menu except the Dashboards.

Try restoring this option to the default value.

  # kibana_all_read kibanauser
    opensearch_security.readonly_mode.roles: [ "kibana_read_only" ]

Hi @pablo ,

Now I can see more yes thank you. Only no indices (data) yet.
Which place is the best to give this kibana_user with backendrole kibanauser access to these indices?

@Raki You should give kibana_user access to any user that accesses OpenSearch Dashboards UI.
What indices are you referring to?

yes accessing dashboard is oke now.
but it seems kibana_user is not allowed to see all our indices such as the filebeat ones.


@pablo , basically I need to set these index_permissions for this role/user

and I would like to it other than using the api like:

curl --insecure -XGET -k -u <uname:pw> https://opensearch-cluster-data.monitoring.svc:9200/_plugins/_security/api/roles/kibana_user         

{"kibana_user":{"reserved":true,"hidden":false,"description":"Provide the minimum permissions for a kibana user","cluster_permissions":["cluster_composite_ops"],"index_permissions":[{"index_patterns":[".kibana",".kibana-6",".kibana_*",".opensearch_dashboards",".opensearch_dashboards-6",".opensearch_dashboards_*"],"fls":[],"masked_fields":[],"allowed_actions":["read","delete","manage","index"]},{"index_patterns":[".tasks",".management-beats","*:.tasks","*:.management-beats"],"fls":[],"masked_fields":[],"allowed_actions":["indices_all"]}],"tenant_permissions":[],"static":true}}

@Raki I think this is the best place to start with roles creation.

Just bear in mind that you have three ways to set and manage the roles.

  1. OpenSearch Dashboards
  2. YML files and securityadmin.sh script
  3. REST APIs

@pablo I think I am missing something.
I tried with the api PUT function to overwrite the kibana_user by adding in e.g. index filebeat-* like so:

curl -k -u {{ op_user }}:{{ op_pass }} -X PUT -H 'Content-Type: application/json' 'https://localhost:9200/_plugins/_security/api/roles/kibana_user' -d '{
    "reserved": true,
    "hidden": false,
    "description": "Provide the minimum permissions for a kibana user",
    "cluster_permissions": [
      "cluster_composite_ops"
    ],
    "index_permissions": [
      {
        "index_patterns": [
          ".kibana",
          ".kibana-6",
          ".kibana_*",
          ".opensearch_dashboards",
          ".opensearch_dashboards-6",
          ".opensearch_dashboards_*",
          "logstash-*",
          "filebeat-*",
          "journalbeat-*",
          "audit-*"
        ],
        "fls": [],
        "masked_fields": [],
        "allowed_actions": [
          "read",
          "delete",
          "manage",
          "index"
        ]
      },
    ],
    "tenant_permissions": [],
    "static": true
}'

But logging in as admin I dont see the indices added in there.

Also I had already read that page.
What I am missing is to be able to set the allowed indices field in the roles_mapping.yml with a field like for example:

        kibana_server:
          reserved: true
          hidden: false
          backend_roles: []
          hosts: []
          users:
          - "kibanaserver"
          and_backend_roles: []
# and something like
          allowed_indices: ["filebeat-*", "foobar-*"]

So from your response I need option 2, preferably not option 3 and surely not option 1.

@Raki roles_mapping.yml is only for mapping backend roles or users with existing roles.
Access to indices is configured only in roles.

“index_patterns” only defines the patterns of existing or future indices in the cluster. This is not equal to creating the indices.

hi @pablo ,

oke so now I have the following config in my nodes:

sh-4.2$ cat config/opensearch-security/roles.yml 
_meta:
  type: "roles"
  config_version: 2
op_dashboard_user:
  reserved: false
  hidden: false
  cluster_permissions:
  - "cluster_composite_ops"
  - "read"
  index_permissions:
  - index_patterns:
    - "logstash-*"
    - "filebeat-*"
    - "journalbeat-*"
    - "audit-*"
    fls: []
    masked_fields: []
    allowed_actions: [
      "read",
      "delete",
      "manage",
      "index"
    ]
  tenant_permissions: []
  static: false
sh-4.2$ cat config/opensearch-security/roles_mapping.yml 
_meta:
  type: "rolesmapping"
  config_version: 2
all_access:
  reserved: false
  backend_roles:
    - "admin"
  users: []
  description: "Maps admin to all_access"
readall:
  reserved: false
  hidden: false
  backend_roles:
  - "readall"
  hosts: []
  users: []
  and_backend_roles: []
logstash:
  reserved: false
  hidden: false
  backend_roles:
  - "logstash"
  hosts: []
  users:
    - logstash
  and_backend_roles: []
kibana_server:
  reserved: true
  hidden: false
  backend_roles: []
  hosts: []
  users:
  - "kibanaserver"
  and_backend_roles: []
own_index:
  reserved: false
  hidden: false
  backend_roles: []
  hosts: []
  users:
  - "*"
  and_backend_roles: []
  description: "Allow full access to an index named like the username"
kibana_user:
  reserved: false
  hidden: false
  backend_roles: []
  hosts: []
  users: []
  and_backend_roles: []
  description: "Maps kibanauser to kibana_user"
op_dashboard_user:
  reserved: false
  hidden: false
  backend_roles:
  - "kibanauser"
  hosts: []
  users: []
  and_backend_roles: []
  description: "Maps kibanauser to opensearch_dashboard_user"

so I overwrote the kibana_user (I tried updating this user but I got an error) and I remapped the kibanauser backend role to a new role op_dashboard_user. I expect to see my filebeat indices, which are visible see when giving myself admin rights, but I still dont. See images. What did i miss?

op_dasbhoard_user rights:


admin rights:


@Raki You’re mixing index with OpenSearch Dashboards index pattern.
An index contains the data. Index pattern filters existing indices.

Even if you have access to your index it doesn’t mean you’ll be able to create the index pattern.
You must create index permission indices:admin/resolve/index for all indices * and assign it to your custom user to enable index pattern creation.

That solved it, now I have a system to go to production with!
Thank you so much for all your help. Appreciated!