Too many redirects - opster opensearch - keycloak - openid 401 unauthorized

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch chart version: 3.0.0
opensearch image version: 2.15.0
opensearch dashboard image version: 2.18.0
operator version: 2.6.0

Describe the issue:
Getting too many redirects error when authenticating opensearch dashboards via keycloak.
Few points to highlight:

  • the same setup was working earlier when we deployed opensearch using opster. Now, we have switched to helm-charts and getting this error.
    only config change passed to opensearch cluster:
cluster.general.setVMMaxMapCount is set to false to resolve below error:

Privileged container is not allowed: init-sysctl, securityContext: {"privileged": true}
"validation.gatekeeper.sh" denied the request: [azurepolicy-k8sazurev3noprivilegeescalatio-eff45a845869734981b7] Privilege escalation container is not allowed

not sure if it was correct or not

  • realm role assigned to client matches with the one configured in securityconfig.
  • ingress has annotation for increasing proxy-buffer-size set to 256k
  • redirect url configured in keycloak is same as of dashboard with endpoint auth/openid/login added.
  • all urls use https
  • for secure communication tls is setup for dashboard and keycloak i.e
dashboards:
    tls:
                enable: true
                generate: false
                secret: 
                    name: "dashboard-secret"

Configuration:
following is opensearch.yaml configuration:

logging.verbose: "true"
server.ssl.certificateAuthorities: "/usr/share/opensearch/config/ca.crt"
opensearch_security.openid.scope: "openid email profile"
opensearch.ssl.verificationMode: full
opensearch.ssl.certificateAuthorities: "/usr/share/opensearch/config/ca.crt"
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://keycloak_url/auth/realms/dedicated/.well-known/openid-configuration"
opensearch_security.openid.client_id: "opensearch-client"
opensearch_security.openid.client_secret: "secret"
opensearch_security.openid.base_redirect_url: "https://opensearch-dashboard-url"
opensearch_security.openid.root_ca: "/usr/share/opensearch/config/ca.crt"
opensearch_security.openid.trust_dynamic_headers: "true"  
opensearch.requestHeadersAllowlist: "security_tenant Authorization x-forwarded-for"

here is security config

roles.yml: |-
  _meta:
      type: "roles"
      config_version: 2
  test-role:
      reserved: true
      hidden: false
      cluster_permissions:
      - "cluster:admin/opendistro/alerting/alerts/ack"
      - "cluster:admin/opendistro/alerting/alerts/get"
      - "cluster:monitor/*"
      - "indices:data/read/*"
      index_permissions:
      - index_patterns:
        - "*"
        allowed_actions:
        - "indices:admin/template/get"
        - "indices:admin/template/put"
        - "indices:data/read/*"
        - "indices:monitor/*"
        - "indices:admin/get"
        - "indices:admin/mappings/get"
        - "indices:admin/aliases/get"
roles_mapping.yml: |-
  _meta:
    type: "rolesmapping"
    config_version: 2

  # Define your roles mapping here

  ## Demo roles mapping

  all_access:
    reserved: false
    users:
    - realm-admin
    backend_roles:
    - "admin"
    description: "Maps admin to all_access"
  
  test-role:
    reserved: false
    hidden: false
    hosts: []
    users:
    - "*"

  own_index:
    reserved: false
    users:
    - "*"
    description: "Allow full access to an index named like the username"

  logstash:
    reserved: false
    backend_roles:
    - "logstash"

  kibana_user:
    reserved: false
    backend_roles:
    - "kibanauser"
    description: "Maps kibanauser to kibana_user"
  
  readall:
    reserved: false
    users:
     - "*"
    backend_roles:
    - "readall"
    
  manage_snapshots:
    reserved: false
    backend_roles:
    - "snapshotrestore"

  kibana_server:
    reserved: true
    users:
    - "kibanaserver"
config.yml: |-
  _meta:
    type: "config"
    config_version: 2
  config:
    dynamic:
          do_not_fail_on_forbidden: true
          http:
            anonymous_auth_enabled: false
          authc:
            basic_internal_auth_domain:
              description: "Authenticate via HTTP Basic"
              http_enabled: true
              transport_enabled: true
              order: 0
              http_authenticator:
                type: "basic"
                challenge: false
              authentication_backend:
                type: "internal"

            openid_auth_domain:
              http_enabled: true
              transport_enabled: true
              order: 1
              http_authenticator:
                type: openid
                challenge: true
                config:
                  openid_connect_idp:
                    enable_ssl: true
                    verify_hostnames: false
                    # pemtrustedcas_filepath: "/usr/share/opensearch/config/ca.crt"
                    pemtrustedcas_content: |
                        root_ca
                  subject_key: preferred_username
                  roles_key: opensearch_roles
                  openid_connect_url: "https://keycloak-url/auth/realms/dedicated/.well-known/openid-configuration"
              authentication_backend:
                type: noop

Relevant Logs or Screenshots:
in keycloak, following roles are assigned to opensearch-client

error seen in dahsboard logs:

{"type":"log","@timestamp":"2025-02-13T12:13:42Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Response Error: 401 Unauthorized"}

in master node:

[2025-02-13T11:42:20,973][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [apm-uat-opensearch-masters-1] Failed to get roles from JWT claims with roles_key 'opensearch_roles'. Check if this key is correct and available in the JWT payload.
[2025-02-13T11:42:21,518][WARN ][o.o.s.h.HTTPBasicAuthenticator] [apm-uat-opensearch-masters-1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'

jwt token value for client:

"realm_access": {
    "roles": [
      "uma_authorization",
      "default-roles-dedicated"
    ]
  },
  "resource_access": {
    "realm-management": {
      "roles": [
        "view-identity-providers",
        "view-realm",
        "manage-identity-providers",
        "impersonation",
        "realm-admin",
        "create-client",
        "manage-users",
        "query-realms",
        "view-authorization",
        "query-clients",
        "query-users",
        "manage-events",
        "manage-realm",
        "view-events",
        "view-users",
        "view-clients",
        "manage-authorization",
        "manage-clients",
        "query-groups"
      ]
    },
    "opensearch-client": {
      "roles": [
        "uma_protection"
      ]
    },
    "account": {
      "roles": [
        "manage-account",
        "manage-account-links",
        "view-profile"
      ]
    }
  },
  "scope": "email opensearch_roles profile",
  "email_verified": false,
  "clientHost": "x.x.x.x",
  "opensearch_roles": [
    "uma_authorization",
    "default-roles-dedicated"
  ],

Hi @Ghata,

I do not see roles from opensearch_roles mapped to Opensearch`s roles or permissions - as per above.

How do you map your JWT key “opensearch_roles” to permissions on your Opensearch end?

Best,
mj

Hi @Mantas ,

I didn’t understand your question. what should be the setup be like?

the role “uma_authorization” is a realm-role and assigned to client-scope opensearch_roles in keycloak and then using roles_key, it is assigned to opensearch`s role.

Is it not correct? if not, then what should be the flow?

Regards
Ghata

in your config.yml you have to define roles_key values that will be used as a back-end roles:
in your case

this is a key in your jwt:

So in this case, your user back-end roles are [“uma_authorization”, “default-roles-dedicated”].
these then need to be mapped to your OpenSearch roles: Defining users and roles - OpenSearch Documentation

best,
mj