OpenID Keycloak working with OpenSearch 2.11.1 but not passing roles

Opensearch version: 2.11.1
Keycloak version: 21.1.2

I’m using opensearch on VM via ansible (copy_custom_security_configs: false), Dashboard via helm (behind ingress-nginx k8s).
Authorization via openid keycloak work, but not passing roles(?).
What did I miss?

Configuration Security Opensearch:

{
    "config": {
        "dynamic": {
            "filtered_alias_mode": "warn",
            "disable_rest_auth": false,
            "disable_intertransport_auth": false,
            "respect_request_indices_options": false,
            "kibana": {
                "multitenancy_enabled": true,
                "private_tenant_enabled": true,
                "default_tenant": "",
                "server_username": "kibanaserver",
                "index": ".kibana"
            },
            "http": {
                "anonymous_auth_enabled": false,
                "xff": {
                    "enabled": true,
                    "internalProxies": ".*",
                    "remoteIpHeader": "x-forwarded-for"
                }
            },
            "authc": {
                "openid_auth_domain": {
                    "http_enabled": true,
                    "transport_enabled": true,
                    "order": 1,
                    "http_authenticator": {
                        "challenge": false,
                        "type": "openid",
                        "config": {
                            "enable_ssl": true,
                            "verify_hostnames": true,
                            "pemtrustedcas_filepath": "/usr/share/opensearch/config/root-ca-keycloak.pem",
                            "subject_key": "preferred_username",
                            "roles_key": "roleopen",
                            "openid_connect_url": "https://{{ keycloak_url }}/realms/{{ realm }}/.well-known/openid-configuration",
                            "kibana_url": "https://{{ dashboard-k8s }}"
                        }
                    },
                    "authentication_backend": {
                        "type": "noop",
                        "config": {}
                    },
                    "description": "Authenticate via OpenID"
                },
                "basic_internal_auth_domain": {
                    "http_enabled": true,
                    "transport_enabled": false,
                    "order": 0,
                    "http_authenticator": {
                        "challenge": false,
                        "type": "basic",
                        "config": {}
                    },
                    "authentication_backend": {
                        "type": "internal",
                        "config": {}
                    },
                    "description": "Authenticate via HTTP Basic against internal users database"
            },
            "authz": {},
            "auth_failure_listeners": {},
            "do_not_fail_on_forbidden": false,
            "multi_rolespan_enabled": true,
            "hosts_resolver_mode": "ip-only",
            "do_not_fail_on_forbidden_empty": false
        }
    }
}
opensearch.yml
cluster.name: "development-cluster"
node.name: "os1"
network.host: "10.0.0.14"
http.port: 9200
discovery.type: single-node
bootstrap.memory_lock: true
plugins.security.allow_default_init_securityindex: true
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.unsupported.restapi.allow_securityconfig_modification: true
plugins.security.ssl.transport.pemcert_filepath: os1.pem
plugins.security.ssl.transport.pemkey_filepath: os1.key
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: os1_http.pem
plugins.security.ssl.http.pemkey_filepath: os1_http.key
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.nodes_dn:
- CN=os1.{{ opensearch }},OU=Ops,O={{ opensearch }}\,
  Inc.,DC={{ opensearch }}
plugins.security.authcz.admin_dn:
- CN=admin.{{ opensearch }},OU=Ops,O={{ opensearch }}\,
  Inc.,DC=mydomain
## END opensearch Security Node & Admin certificates configuration ##
Values helm chart dashboard
opensearchHosts: "https://{{ opensearch }}:9200"
replicaCount: 1

image:
  repository: "opensearchproject/opensearch-dashboards"
  tag: "2.4.0"
  pullPolicy: "IfNotPresent"

startupProbe:
  tcpSocket:
    port: 5601
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 20
  successThreshold: 1
  initialDelaySeconds: 10

livenessProbe:
  tcpSocket:
    port: 5601
  periodSeconds: 20
  timeoutSeconds: 5
  failureThreshold: 10
  successThreshold: 1
  initialDelaySeconds: 10

readinessProbe:
  tcpSocket:
    port: 5601
  periodSeconds: 20
  timeoutSeconds: 5
  failureThreshold: 10
  successThreshold: 1
  initialDelaySeconds: 10

imagePullSecrets: []
nameOverride: "opensearch-dashboard"
fullnameOverride: "opensearch-dashboard-web"

serviceAccount:
  create: true
  annotations: {}
  name: "opensearch-dashboard"

rbac:
  create: true

secretMounts: []

podAnnotations: {}

dashboardAnnotations: {}

extraEnvs: []

envFrom: []

extraVolumes: []

extraVolumeMounts: []

extraInitContainers: ""

extraContainers: ""

podSecurityContext: {}

securityContext:
  capabilities:
    drop:
      - ALL
  # readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000

config:
   opensearch_dashboards.yml: |
    server.host: "{{ .Values.serverHost }}"
      # Enable OpenID authentication
    opensearch.requestHeadersAllowlist: ["securitytenant","Authorization","x-forwarded-for"]
    opensearch.ssl.verificationMode: none
    opensearch_security.cookie.secure: false
    *# opensearch_security.auth.type: openid*
    opensearch_security.openid.base_redirect_url: "{{ dashboard-k8s }}"
    opensearch_security.openid.client_id: "{{ client_id }}"
    opensearch_security.openid.scope: "email profile roleopen groups"
    opensearch_security.openid.client_secret: "secret"
    opensearch_security.openid.connect_url: "https://{{ keycloak_url }}/realms/{{ realm }}/.well-known/openid-configuration"
    opensearch_security.openid.header: "Authorization"
    opensearch_security.openid.verify_hostnames: false
    *# opensearch.ssl.certificate: /path/to/your/client.crt*
*    # opensearch.ssl.key: /path/to/your/client.key*
*    # opensearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]*
    opensearch_security.auth.type: ["basicauth","openid"]
    opensearch_security.auth.multiple_auth_enabled: true

opensearchDashboardsYml:
  defaultMode:
  # value should be 0-0777

priorityClassName: ""

opensearchAccount:
  secret: "opensearchsecretconn"
  keyPassphrase:
    enabled: false

labels: {}

hostAliases: []

serverHost: "0.0.0.0"

service:
  type: ClusterIP
  port: 5601
  loadBalancerIP: "10.4.0.25"
  nodePort: ""
  labels: {}
  annotations: {}
  loadBalancerSourceRanges: []
  # 0.0.0.0/0
  httpPortName: http

ingress:
  enabled: **true**
  # For Kubernetes >= 1.18 you should specify the ingress-controller via the field ingressClassName
  # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
  ingressClassName: opensearch-dashboard-web
  annotations:
    meta.helm.sh/release-name: opensearch
    meta.helm.sh/release-namespace: opensearch-stage
    nginx.org/location-snippets: |
      add_header X-Robots-Tag "noindex, nofollow" always;
  labels: 
    app.kubernetes.io/instance: opensearch-stage
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: opensearch
    helm.sh/chart: opensearch-2.11.1
  hosts:
    - host: {{ dashboard-k8s }}
      paths:
        - path: /
          backend:
            serviceName: ""
            servicePort: ""
  tls:
    - secretName: k8s-secret
      hosts:
        - {{ dashboard-k8s }}

ingressClass:
  enabled: true
  controller: k8s.io/ingress-nginx
  annotations: {}
  labels:
    app.kubernetes.io/component: controller

resources:
  requests:
    cpu: "100m"
    memory: "512M"
  limits:
    cpu: "1000m"
    memory: "2048M"

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 10
  targetCPUUtilizationPercentage: 80

updateStrategy:
  type: "Recreate"

nodeSelector: {}

tolerations: []

affinity: {}

extraObjects: []
lifecycle: {}
plugins:
  enabled: false
  installList: []

Opensearch dashboard backend role:

Authorization via openid OK, but not passing roles from Keycloak:

In opensearch logs:

[2023-12-29T18:32:42,077][WARN ][o.o.s.h.HTTPBasicAuthenticator] [os1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-12-29T18:32:42,077][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [os1] Failed to get roles from JWT claims with roles_key 'roleopen'. Check 
if this key is correct and available in the JWT payload.

In Opensearch Dashboard Pod:

{"type":"log","@timestamp":"2023-12-29T18:32:39Z","tags":["error","opensearch","data"],"pid":1,"message":"[security_exception]: no permissions for [indices:data/read/get] and User [name=someuser, backend_roles=[], requestedTenant=null]"}

Keycloak configuration Client, Tab Settings:

Access settings

Capability config

Keycloak configuration Client, Tab Keys:

JWKS URL configs

Use JWKS URL=off

Keycloak configuration Client, Tab Credentials:

Client Authenticator

Client Id and secret with some client secret

Registration access token

null

Keycloak configuration Client, Tab Roles:

Role name

roles

Keycloak configuration Client, Tab Client scopes:

Assigned client scope

Keycloak configuration Client scopes

roleopen

Tab Settings:


Tab Mappers

realm roles

client roles

Hi @ActiveSync ,

Could you try setting "Token Claim Name" to roles instead of realm_access.roles and "Add to userinfo" ON

Best,
mj

Hi @Mantas. It works! Thanks :slight_smile:

After authorization I got 502 bad gateway error (I use ingress-nginx k8s).
Solution here:
Proxy_buffer_size : 502 bad gateway error - Getting advice - Keycloak

1 Like