OpenID(keycloak) 401 Unauthorized

jun · December 12, 2022, 8:13am

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch - 2.4.0 / Dashboard - 2.4.0 /Server OS - Centos 7.9 / Browser - Chrome

Describe the issue:

I’m testing opensearch 2.4.0 cluster and Opensearch on VM server and installed opensearch from a tarball.

I’m trying to integrate with OpenID Connect by following the guide here ([OpenID Connect - Documentation]). But never allows me to visit login page, just straight error:
{"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

Configuration:

config.yml authc:

    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: "internal"
    openid_auth_domain:
      http_enabled: false
      transport_enabled: true
      order: 1
      http_authenticator:
        type: openid
        challenge: false
        config:
          subject_key: preferred_username
#          subject_key: given_name
          roles_key: roles
          openid_connect_url: https://dev-sso.cloud.hyundai-autoever.com/auth/realms/HCP/.well-known/openid-configuration
      authentication_backend:
        type: noop

opensearch_dashboard.yml

opensearch.hosts: [https://10.5.186.236:9200]
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: ["Authorization", "securitytenant"]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]
opensearch_security.cookie.secure: false


opensearch_security.auth.type: ["basicauth","openid"]
opensearch_security.auth.multiple_auth_enabled: true
server.xsrf.allowlist: ["/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]



opensearch_security.openid.connect_url: "https://my-keycloak/ auth/realms/HCP/.well-known/openid-configuration"
opensearch_security.openid.client_id: "opensearch"
opensearch_security.openid.client_secret: "NWqUEsgkRGvrggUhD6EPGEt5NfCjg2wV"
opensearch_security.openid.base_redirect_url: "http://10.5.186.236:5601/api/security/v1/master"
opensearch_security.openid.scope: "openid profile email"
opensearch_security.openid.header: "Authorization"

Keyclaok Settings

Relevant Logs or Screenshots:
Dashboard logs

  log   [07:32:02.293] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [07:32:02.293]  GET /favicon.ico 401 1ms - 9.0B
  log   [07:32:06.479] [debug][metrics] Refreshing metrics

when I excute the security_admin.sh, generated Error log as below.

/data/opensearch/plugins/opensearch-security/tools/securityadmin.sh \
  -cacert /data/opensearch/config/root-ca.pem \
  -cert /data/opensearch/config/kirk.pem \
  -key /data/opensearch/config/kirk-key.pem \
  -cd /data/opensearch/config/opensearch-security/

log

Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.4.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: test
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /data/opensearch/config/opensearch-security/
ERR: Seems /data/opensearch/config/opensearch-security/config.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "openid_auth_domain" (class org.opensearch.security.securityconf.impl.v7.ConfigV7$Dynamic), not marked as ignorable (15 known properties: "license", "disable_intertransport_auth", "http", "kibana", "authz", "auth_failure_listeners", "transport_userrname_attribute", "filtered_alias_mode", "authc", "disable_rest_auth", "respect_request_indices_options", "multi_rolespan_enabled", "do_not_fail_on_forbidden", "hosts_resolver_mode", "do_not_fail_on_forbidden_empty"])
 at [Source: (String)"{"_meta":{"type":"config","config_version":2},"config":{"dynamic":{"http":{"anonymous_auth_enabled":false,"xff":{"enabled":false,"internalProxies":"192\\.168\\.0\\.10|192\\.168\\.0\\.11"}},"authc":{"basic_internal_auth_domain":{"description":"Authenticate via HTTP Basic against internal users database","http_enabled":true,"transport_enabled":true,"order":0,"http_authenticator":{"type":"basic","challenge":false},"authentication_backend":{"type":"internal"}}},"openid_auth_domain":{"http_enabled":f"[truncated 1172 chars]; line: 1, column: 485] (through reference chain: org.opensearch.security.securityconf.impl.SecurityDynamicConfiguration["config"]->org.opensearch.security.securityconf.impl.v7.ConfigV7["dynamic"]->org.opensearch.security.securityconf.impl.v7.ConfigV7$Dynamic["openid_auth_domain"])
ERR: Seems /data/opensearch/config/opensearch-security/roles.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "cluster" (class org.opensearch.security.securityconf.impl.v7.RoleV7), not marked as ignorable (7 known properties: "index_permissions", "reserved", "hidden", "description", "static", "cluster_permissions", "tenant_permissions"])
 at [Source: (String)"{"_meta":{"type":"roles","config_version":2},"kibana_read_only":{"reserved":true},"security_rest_api_access":{"reserved":true},"alerting_read_access":{"reserved":true,"cluster_permissions":["cluster:admin/opendistro/alerting/alerts/get","cluster:admin/opendistro/alerting/destination/get","cluster:admin/opendistro/alerting/monitor/get","cluster:admin/opendistro/alerting/monitor/search","cluster:admin/opensearch/alerting/findings/get"]},"alerting_ack_alerts":{"reserved":true,"cluster_permissions":"[truncated 7235 chars]; line: 1, column: 7659] (through reference chain: org.opensearch.security.securityconf.impl.SecurityDynamicConfiguration["sg_human_resources"]->org.opensearch.security.securityconf.impl.v7.RoleV7["cluster"])
ERR: Seems /data/opensearch/config/opensearch-security/roles_mapping.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "backendroles" (class org.opensearch.security.securityconf.impl.v7.RoleMappingsV7), not marked as ignorable (7 known properties: "backend_roles", "users", "hidden", "hosts", "description", "reserved", "and_backend_roles"])
 at [Source: (String)"{"_meta":{"type":"rolesmapping","config_version":2},"all_access":{"reserved":false,"backend_roles":["admin"],"description":"Maps admin to all_access"},"own_index":{"reserved":false,"users":["*"],"description":"Allow full access to an index named like the username"},"logstash":{"reserved":false,"backend_roles":["logstash"]},"kibana_user":{"reserved":false,"backend_roles":["kibanauser"],"description":"Maps kibanauser to kibana_user"},"readall":{"reserved":false,"backend_roles":["readall"]},"manage"[truncated 171 chars]; line: 1, column: 665] (through reference chain: org.opensearch.security.securityconf.impl.SecurityDynamicConfiguration["sg_human_resources"]->org.opensearch.security.securityconf.impl.v7.RoleMappingsV7["backendroles"])
Will update '/internalusers' with /data/opensearch/config/opensearch-security/internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /data/opensearch/config/opensearch-security/action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /data/opensearch/config/opensearch-security/tenants.yml
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /data/opensearch/config/opensearch-security/nodes_dn.yml
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /data/opensearch/config/opensearch-security/whitelist.yml
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /data/opensearch/config/opensearch-security/audit.yml
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /data/opensearch/config/opensearch-security/allowlist.yml
   SUCC: Configuration for 'allowlist' created or updated
ERR: cannot upload configuration, see errors above

Please help what the problem…

pablo · December 12, 2022, 9:08am

@jun What is the Keycloak’s version?
According to your last output, the securityadmin.sh failed to update config.yml.
As far as I can see, openid_auth_domain has incorrect indent and is at the same level as authc.

pablo · December 12, 2022, 12:58pm

@jun In your config.yml you have roles_key: roles but the Token Claim Name in the Mappers is all_access. These values have to match to pass the roles from the keycloak.

Also, the below is incorrect.

opensearch_security.openid.base_redirect_url: "http://10.5.186.236:5601/api/security/v1/master"

It should be.

opensearch_security.openid.base_redirect_url: "http://10.5.186.236:5601"

jun · December 13, 2022, 1:54am

Hi, pablo.
My keycloak version is 18.0.0.
After modified indent in config.yml as your guide , securityadmin.sh was executed successfully
and changed opensearch_dashboard.yml

opensearch_security.openid.base_redirect_url: "http://10.5.186.236:5601"

But still having the same error.

opensearch.log

[2022-12-13T10:50:59,172][WARN ][o.o.s.h.HTTPBasicAuthenticator] [node-1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'

opensearch_dashboard

 [error][plugins][securityDashboards] OpenId authentication failed: Error: Response Error: 400 Bad Request
respons [01:54:26.167]  GET /auth/openid/login?state=O14wF3BZAI2ueU-GXIwv0G&session_state=624a14be-c91b-473d-9c09-fe2a7f360754&code=b6b9aca8-07c6-4dd6-a55c-62967580a29a.624a14be-c91b-473d-9c09-fe2a7f360754.69979055-ff4d-4f16-9b31-bf658bb9fb06 302 11ms - 9.0B

jun · December 13, 2022, 1:55am

Could you guide how to match the roles_key and Mappers?

pablo · December 14, 2022, 12:23pm

@jun Your role mapper is correct. However, you need to change the value of the Token Claim Name to roles so it will correspond with roles_key: roles in config.yml or change roles_key to roles_key: all_access.
Then you need to create roles and assign to the users in Keycloak. Finally assign those roles in the roles_mapping.yml as backend_roles.

gehf · December 14, 2022, 12:42pm

You can reference my thread:

please check whether registering route handler for [/auth/openid/login] successful before http server running at https://0.0.0.0:5601.

As I test, the openid route handler is not register successful before the server start. So it will always 401.

jun · December 15, 2022, 8:48am

Thank you for your reply.

But I don’t know how to assign the keycloak users and setting the roles_mapping.yml properly.

Any examples of how to configure it?

jun · December 15, 2022, 8:53am

Thanks for you reply.
I checked the /auth/openid/login registed log line as below.

 log   [01:48:24.402] [debug][server][OpenSearchDashboards][http] registering route handler for [/auth/openid/login]
  log   [01:48:24.402] [debug][server][OpenSearchDashboards][http] registering route handler for [/auth/openid/logout]

But still having the same error.

opensearch.log

[2022-12-15T10:50:59,172][WARN ][o.o.s.h.HTTPBasicAuthenticator] [node-1] No 'Basic Authorizat

Anything else to check?

pablo · December 16, 2022, 6:06pm

@jun Please follow this documentation.

jun · January 10, 2023, 5:54am

@pablo
hi,

After I set the config.yml and keycloak as your guide, the error message “No ‘Basic Authorization’ header, send 401 and ‘WWW-Authenticate Basic’” was solved.
But when I try to connect from OIDC , generated Error log as below and back to log in page(http://10.5.186.236:5601/app/login?)

respons [05:53:48.195]  GET /app/login 200 9ms - 9.0B
  log   [05:53:48.275] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [05:53:48.274] [api] GET /bootstrap.js 304 5ms - 9.0B
respons [05:53:49.123]  GET /translations/en.json 304 1ms - 9.0B
respons [05:53:49.126]  GET /ui/legacy_light_theme.css 200 4ms - 9.0B
respons [05:53:49.131]  GET /node_modules/@osd/ui-framework/dist/kui_light.css 200 2ms - 9.0B
  log   [05:53:49.415] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [05:53:49.415]  GET /api/v1/restapiinfo 401 1ms - 9.0B
respons [05:53:49.417]  GET /ui/fonts/inter_ui/Inter-UI-Regular.woff2 200 1ms - 9.0B
  log   [05:53:49.445] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [05:53:49.445]  GET /api/v1/configuration/account 401 1ms - 9.0B
respons [05:53:49.464]  POST /api/core/capabilities 200 3ms - 9.0B

JWT token samle was as bellow.

{
  "exp": 1673350065,
  "iat": 1673335665,
  "jti": "7c4b929f-44b5-4844-8ccd-ac02a8abbff3",
  "iss": "https://mycompany/auth/realms/TEST",
  "aud": [
    "test",
    "account"
  ],
  "sub": "43b2cf9f-92f6-496d-985c-84e7f1413ee7",
  "typ": "Bearer",
  "azp": "opensearch",
  "session_state": "ae1ac5c1-300a-4ecb-920e-6df802634a37",
  "acr": "1",
  "realm_access": {
    "roles": [
      "DEFAULT-MEMBER",
      "offline_access",
      "uma_authorization",
      "COMPANY-MEMBER"
    ]
  },
  "resource_access": {
    "account": {
      "roles": [
        "manage-account",
        "manage-account-links",
        "view-profile"
      ]
    }
  },
  "scope": "openid displayName department app-audience groups profile company email",
  "sid": "ae1ac5c1-300a-4ecb-920e-6df802634a37",
  "email_verified": false,
  "displayName": "test",
  "roles": [
    "DEFAULT-MEMBER",
    "offline_access",
    "uma_authorization",
    "COMPANY-MEMBER"
  ],
  "name": "test",
  "groups": [
    "/test",
    "/DEFAULT-GROUP"
  ],
  "company": "test-company",
  "preferred_username": "test",
  "department": "test; testteam",
  "given_name": "test",
  "family_name": "test",
  "email": "kwonhc@gmail.com"
}

Topic		Replies	Views
Keycloak OpenID 401 Unauthorized Security	22	5595	September 21, 2022
Problems while login via keycloak(OpenID connect) Security	13	102	September 27, 2024
OpenSearch Dashboards 401 Unauthorized using OIDC with Keycloak Security	12	6471	February 15, 2023
Authorization issue using OpenID (Keycloak) Security	17	1323	November 13, 2023
OpenID 401 Unauthorized with Keycloak Security troubleshoot , configure	14	3136	August 3, 2022

OpenID(keycloak) 401 Unauthorized

Related topics