Infinite redirect loop when trying to secure Opensearch with a keycloak located in the same cluster

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch: 2.0.0
Opensearch-dashboard: 2.0.0
Keycloak: 21.1.2
Server: Ubuntu 22.04 x64
Browser: Edge 116.0.1938.54 and Chrome 115.0.5790.66

Describe the issue:
I’m working on a POC to secure dashboards in my Microk8s cluster, including Opensearch Dashboard, using a Keycloak located in the same cluster. I’m using OpenID Connect.
My Opensearch Dashboard is accessible using https://cluster_host/opensearch-dashboards/ and my keycloak server is accessible using https://cluster_host/keycloak/.

I had a first issue because it looks that depending on the phase of the authentication, there is some communication inside the cluster (directly between osd and keycloak) and outside the cluster (with my browser).
To solve this, I modified the CoreDNS configuration to add the line

rewrite name cluster_host keycloak.default.svc.cluster.local

Thanks to this, I’m able to access to the keycloak authentication page but when I log in, I have then a ERR_TOO_MANY_REDIRECT

Can anyone tell me where is my mistake? I was wondering if it could be due to the fix I did for the first issue.

Thanks in advance for your help

Configuration:

opensearch_dashboards.yml: 
   server.host: '0.0.0.0'
   server.basePath: "/opensearch-dashboard"
   server.rewriteBasePath: true
   opensearch.hosts: ["https://localhost:9200"]
   opensearch.username: "kibanaserver"
   opensearch.password: "kibanaserver"
   opensearch.requestHeadersAllowlist: ["security_tenant","Authorization","x-forwarded-for"]
   opensearch.ssl.verificationMode: none
   opensearch_security.auth.type: "openid"
   opensearch_security.openid.connect_url: http://cluster_host/keycloak/realms/myrealm/.well-known/openid-configuration
   opensearch_security.openid.base_redirect_url: https://cluster_host/opensearch-dashboard/
   opensearch_security.openid.client_id: opensearch-oauth
   opensearch_security.openid.client_secret: "***************"
   opensearch_security.openid.verify_hostnames: false
   opensearch_security.multitenancy.enabled: true
   opensearch_security.multitenancy.tenants.enable_global: true
   opensearch_security.multitenancy.tenants.enable_private: true
   opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
   opensearch_security.multitenancy.enable_filter: false
   opensearch_security.readonly_mode.roles: [kibana_read_only]
   logging.verbose: true
   opensearch_security.cookie.secure: false

I added 2 mappers as I read it was necessary:

  • preferred_username with a mapper type “User’s full name” (all Add options enabled)

  • roles with a mapper type “User Realm Role” (all Add options enabled)

Relevant Logs or Screenshots:
When I try to log in, here is the log I see in Opensearch Dashboard logs:

{“type”:“response”,“@timestamp”:“2023-09-05T07:29:03Z”,“tags”:,“pid”:1,“method”:“get”,“statusCode”:302,“req”:{“url”:“/”,“method”:“get”,“headers”:{“host”:“cluster_host”,“x-request-id”:“11601df8a6f6681bb853cf1b16c80636”,“x-real-ip”:“10.169.9.34”,“x-forwarded-for”:“10.169.9.34”,“x-forwarded-host”:“cluster_host”,“x-forwarded-port”:“443”,“x-forwarded-proto”:“https”,“x-forwarded-scheme”:“https”,“x-scheme”:“https”,“sec-ch-ua”:“"Chromium";v="116", "Not)A;Brand";v="24", "Microsoft Edge";v="116"”,“sec-ch-ua-mobile”:“?0”,“sec-ch-ua-platform”:“"Windows"”,“upgrade-insecure-requests”:“1”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.54”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“none”,“sec-fetch-mode”:“navigate”,“sec-fetch-user”:“?1”,“sec-fetch-dest”:“document”,“accept-encoding”:“gzip, deflate, br”,“accept-language”:“fr,fr-FR;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6”},“remoteAddress”:“10.1.104.226”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.54”},“res”:{“statusCode”:302,“responseTime”:3,“contentLength”:9},“message”:“GET / 302 3ms - 9.0B”}
{“type”:“response”,“@timestamp”:“2023-09-05T07:29:03Z”,“tags”:,“pid”:1,“method”:“get”,“statusCode”:302,“req”:{“url”:“/auth/openid/login”,“method”:“get”,“headers”:{“host”:“cluster_host”,“x-request-id”:“3de79954d032043bc5a2b23c37f34dcb”,“x-real-ip”:“10.169.9.34”,“x-forwarded-for”:“10.169.9.34”,“x-forwarded-host”:“cluster_host”,“x-forwarded-port”:“443”,“x-forwarded-proto”:“https”,“x-forwarded-scheme”:“https”,“x-scheme”:“https”,“upgrade-insecure-requests”:“1”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.54”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“none”,“sec-fetch-mode”:“navigate”,“sec-fetch-user”:“?1”,“sec-fetch-dest”:“document”,“sec-ch-ua”:“"Chromium";v="116", "Not)A;Brand";v="24", "Microsoft Edge";v="116"”,“sec-ch-ua-mobile”:“?0”,“sec-ch-ua-platform”:“"Windows"”,“accept-encoding”:“gzip, deflate, br”,“accept-language”:“fr,fr-FR;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6”},“remoteAddress”:“10.1.104.226”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.54”},“res”:{“statusCode”:302,“responseTime”:1,“contentLength”:9},“message”:“GET /auth/openid/login 302 1ms - 9.0B”}
{“type”:“log”,“@timestamp”:“2023-09-05T07:29:03Z”,“tags”:[“error”,“plugins”,“securityDashboards”],“pid”:1,“message”:“OpenId authentication failed: Error: Authentication Exception”}

I’m continuing to search for an answer on internet and I have the feeling that it could be due to an incorrect JWT token generation (perhaps a mapping is missing) or an issue with a role.

Unfortunatelly, the detailled issue does not appear in my logs, I just have “OpenId authentication failed: Error: Authentication Exception”. Do you know if there is a way to obtain more detailed information?

Thanks in advance for your help!

Try setting this one to

opensearch_security.openid.base_redirect_url: https://cluster_host

and this one to false

server.rewriteBasePath: false

Thank you for your answer.

I tried both proposal separately and together:

  • if I set server.rewriteBasePath to false (with or without the base_redirect_url change), I obtain the following message:

{“statusCode”:401,“error”:“Unauthorized”,“message”:“Unauthorized”}

  • If I change the base_redirect_url as proposed witthout changing the rewriteBasePath value, the following content appears directly (no login page displayed)
    image

@rborredon I think the Unauthorized error is a progress. It means that user has been authenticated but assigned IdP roles either missing or were not mapped in roles_mapping.yml.

Did you configure User Role Realms in the Keycloak’s client?

Well the unauthorized error arrives before any login action. The login page is not displayed in this case.

Regarding the roles, I wanted to use the predefined roles present in Users and roles - OpenSearch documentation.

I only added the all_access role directly in the client (so no in the Realm roles list) and I assigned it directly to my user:

Just for the test, I added the all_access role to the Realm roles list and to my user. Here is the generated token (with my original values of rewritebathpath and base_redirect_url as I cannot log in withtout them):

{
“exp”: 1693953838,
“iat”: 1693953538,
“auth_time”: 0,
“jti”: “99ec9262-eb0b-4efc-8bc3-3246c0734a8a”,
“iss”: “https://cluster_host/keycloak/realms/myrealm”,
“aud”: “opensearch-oauth”,
“sub”: “2176f7d6-8ae7-4999-98ed-e22ce9fcc63d”,
“typ”: “ID”,
“azp”: “opensearch-oauth”,
“session_state”: “7bfdbeff-4eaf-4703-913c-b2be9188e651”,
“acr”: “1”,
“sid”: “7bfdbeff-4eaf-4703-913c-b2be9188e651”,
“clientrole”: “[manage-account, view-applications, view-consent, view-groups, manage-account-links, delete-account, manage-consent, view-profile]”,
“email_verified”: false,
“roles”: [
“default-roles-myrealm”,
“offline_access”,
“all_access”,
“uma_authorization”
],
“name”: “admin admin”,
“groups”: [
“admin”
],
“preferred_username”: “admin”,
“given_name”: “admin”,
“family_name”: “admin”,
“email”: “admin@gmail.com
}

@rborredon Could you used OpenSearch pod name instead of localhost?

I tried your change. The behavior is the same but at least, it is cleaner.

Now, the configuration file is:

 server.host: '0.0.0.0'
 server.basePath: "/opensearch-dashboard"
 server.rewriteBasePath: true
 opensearch.hosts: ["https://cluster_host:9200"]
 opensearch.username: "kibanaserver"
 opensearch.password: "kibanaserver"
 opensearch.requestHeadersAllowlist: ["security_tenant","Authorization","x-forwarded-for"]
 opensearch.ssl.verificationMode: none
 opensearch_security.auth.type: "openid"
 opensearch_security.openid.connect_url: http://cluster_host/keycloak/realms/myrealm/.well-known/openid-configuration
opensearch_security.openid.base_redirect_url: https://cluster_host/opensearch-dashboard/
opensearch_security.openid.client_id: opensearch-oauth
opensearch_security.openid.client_secret: "*************************"
opensearch_security.openid.verify_hostnames: false
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false
opensearch_security.readonly_mode.roles: [kibana_read_only]
logging.verbose: true
opensearch_security.cookie.secure: false

Here is the log generated by y latest connection attempt:

{“type”:“log”,“@timestamp”:“2023-09-06T08:43:15Z”,“tags”:[“error”,“plugins”,“securityDashboards”],“pid”:1,“message”:“OpenId authentication failed: Error: Authentication Exception”}
{“type”:“response”,“@timestamp”:“2023-09-06T08:43:15Z”,“tags”:,“pid”:1,“method”:“get”,“statusCode”:302,“req”:{“url”:“/auth/openid/login?state=ak94VIA0bGGl5tnPrn8S8d&session_state=5154fc90-7bfb-4da8-bf30-ff297e75099b&code=c90f6d96-ef26-40df-8648-b346481bfe4a.5154fc90-7bfb-4da8-bf30-ff297e75099b.18da3c03-db9c-47c3-ab73-ec1240efa3a7”,“method”:“get”,“headers”:{“host”:“cluster_host,“x-request-id”:“565e0fdf382f674e87fb89d73fa79a2f”,“x-real-ip”:“10.21.21.184”,“x-forwarded-for”:“10.21.21.184”,“x-forwarded-host”:“cluster_hostname”,“x-forwarded-port”:“443”,“x-forwarded-proto”:“https”,“x-forwarded-scheme”:“https”,“x-scheme”:“https”,“cache-control”:“max-age=0”,“upgrade-insecure-requests”:“1”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“cross-site”,“sec-fetch-mode”:“navigate”,“sec-fetch-dest”:“document”,“sec-ch-ua”:”"Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"“,“sec-ch-ua-mobile”:”?0",“sec-ch-ua-platform”:“"Windows"”,“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“10.1.104.226”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36”},“res”:{“statusCode”:302,“responseTime”:14,“contentLength”:9},“message”:“GET /auth/openid/login?state=ak94VIA0bGGl5tnPrn8S8d&session_state=5154fc90-7bfb-4da8-bf30-ff297e75099b&code=c90f6d96-ef26-40df-8648-b346481bfe4a.5154fc90-7bfb-4da8-bf30-ff297e75099b.18da3c03-db9c-47c3-ab73-ec1240efa3a7 302 14ms - 9.0B”}