Failing to configure SAML between OpenSearch and Keycloak

Security
1

I am trying to configure SAML between OpenSearch and Keycloak according to this guide.

URLs are as follows:

config.yml

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern

      saml_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: saml
          challenge: true
          config:
            idp:
              metadata_url: https://sso.my-domain.de/auth/realms/MyRealm/protocol/saml/descriptor
              entity_id: https://sso.my-domain.de/auth/realms/MyRealm
            sp:
              entity_id: opensearch_dashboards
            kibana_url: https://logs.my-domain.de
            roles_key: roles
            subject_key: username
            exchange_key: random-exchange-key
        authentication_backend:
          type: noop

opensearch_dashboards.yml

opensearch.requestHeadersWhitelist: ["securitytenant", "Authorization"]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false

opensearch_security.auth.type: "saml"
server.xsrf.whitelist:
  [
    /_plugins/_security/saml/acs,
    /_opendistro/_security/saml/acs,
    /_plugins/_security/saml/acs/idpinitiated,
    /_opendistro/_security/saml/acs/idpinitiated,
    /_plugins/_security/saml/logout,
    /_opendistro/_security/saml/logout,
  ]

The problem is, that I’m not even redirected to Keycloak. There is some error in the Dashboard, not even the first redirect is done:

opensearch-dashboards    | {"type":"response","@timestamp":"2022-06-13T16:19:56Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"logs.my-domain.de","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,de-DE;q=0.7,en;q=0.3","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","te":"trailers","upgrade-insecure-requests":"1","x-forwarded-for":"5.146.48.50","x-forwarded-host":"logs.my-domain.de","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-server":"7599e730a4ee","x-real-ip":"5.146.48.50"},"remoteAddress":"172.18.0.8","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET / 302 5ms - 9.0B"}
opensearch-dashboards    | Error: failed parsing SAML config
opensearch-dashboards    |     at SecurityClient.getSamlHeader (/usr/share/opensearch-dashboards/plugins/securityDashboards/server/backend/opensearch_security_client.ts:176:15)
opensearch-dashboards    |     at runMicrotasks (<anonymous>)
opensearch-dashboards    |     at processTicksAndRejections (internal/process/task_queues.js:95:5)
opensearch-dashboards    |     at /usr/share/opensearch-dashboards/plugins/securityDashboards/server/auth/types/saml/routes.ts:65:30
opensearch-dashboards    |     at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:163:44)
opensearch-dashboards    |     at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)
opensearch-dashboards    |     at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
opensearch-dashboards    |     at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)
opensearch-dashboards    |     at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)
opensearch-dashboards    |     at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)
opensearch-dashboards    |     at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)
opensearch-dashboards    | {"type":"log","@timestamp":"2022-06-13T16:19:56Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"Failed to get saml header: Error: Error: failed parsing SAML config"}
opensearch-dashboards    | {"type":"error","@timestamp":"2022-06-13T16:19:56Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:143:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:97:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:92:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:164:34)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"**http://logs.my-domain.de/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards**","message":"Internal Server Error"}
opensearch-dashboards    | {"type":"response","@timestamp":"2022-06-13T16:19:56Z","tags":[],"pid":1,"method":"get","statusCode":500,"req":{"url":"/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","method":"get","headers":{"host":"logs.my-domain.de","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,de-DE;q=0.7,en;q=0.3","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","te":"trailers","upgrade-insecure-requests":"1","x-forwarded-for":"5.146.48.50","x-forwarded-host":"logs.my-domain.de","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-server":"7599e730a4ee","x-real-ip":"5.146.48.50"},"remoteAddress":"172.18.0.8","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"},"res":{"statusCode":500,"responseTime":37,"contentLength":9},"message":"GET /auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards 500 37ms - 9.0B"}

Two things strike me as odd:

Error: failed parsing SAML config is the only error message I get. Where does the dashboard know which config to fetch in the first place?

Secondly, at the end of the stack trace, it refers to the URL http://logs.my-domain.de**/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards, which is not HTTPs for some reason. Everything has TLS enabled, not sure why http is used there.

I tried to replace metadata_url with metadata_file to avoid any errors when fetching the config, this does not change the behaviour or error messages at all unfortunately.

Any hints where to look? Thanks in advance!

3

@Wasabi
There are a couple of things to check for here.

1st is the https actually turned on in Dashboards? It probably is, but its unclear from the extract in above.

2, Are the certificates in keycloak self-signed / trusted? See below extract for ssl config in saml, notice enable_ssl and pemtrustedcas_filepath:

http_authenticator:
    type: saml
    challenge: true
    config:
        idp:
            enable_ssl: true
            pemtrustedcas_filepath: "/usr/share/opensearch/config/keycloak.crt"
            metadata_url: ....
            entity_id: .....
  1. Lastly, have you confirmed all the details from Keycloak side match up (entity_id, roles_key, subject_key)?
1 Like
4

Hi @Anthony,

thank you for your response!

  1. Dashboards is behind a Traefik reverse proxy, so its not explicitly enabled in Dashboards. I tried setting server.basePath: "https://logs.my-domain.de", but this results in a 404 even when accessing the Dashboards URL. That said, I now notice that basePath is not meant for this, I was looking for a way to define the URL under which OpenSearch is accessed.

  2. Keycloak certificates are Let’s Encrypt certificates and should be trusted also by default JDK settings. I tried to specify metadata_file instead of the URL as well to rule out any connection errors, this didn’t change anything sadly. Currently I’m still using the file.

  3. I verified the entity ID against the value from the XML metadata file, thats correct. The role key is correct as well, I also set flag in the keylcoak mapper to only have one attribute, even if there is multiple roles.

To be fair, this shouldn’t even matter (yet) though, as Dashboards throws the error even before redirecting me to Keycloak. So all the attributes do not come into play (yet).

5

@Wasabi a bit of foundational question, but how are you updating the config settings, are you running the securityadmin.sh script?

Also, have you tried using the below settings? Was there any change?
enable_ssl: true
pemtrustedcas_filepath: …

1 Like
6

Hi @Anthony,

I’m running both in a docker environment (sorry, I somehow forgot to mention that in the initial post), I was under the impression that the security settings are updated automatically when restarting the container. The custom config is mounted into the container and I’m just restarting it when doing any change. So far, all changes became effective after that. Do I need to run something else?

I’ve tried to settings:

            idp:
              metadata_file: /usr/share/opensearch/config/opensearch-security/keycloak.xml
              entity_id: https://sso.my-domain.de/auth/realms/MyTenant

This is essentially the metadata file downloaded, mounted into the container and used directly, just to make sure there’s not any kind of SSL issue. The other option I tried is this one:

          idp:
              metadata_url: https://sso.my-domain.de/auth/realms/MyTenant/protocol/saml/descriptor
              entity_id: https://sso.my-domain.de/auth/realms/MyTenant
              enable_ssl: true
              pemtrustedcas_filepath: "/usr/share/opensearch/config/isrgrootx1.crt"

Both yield the same behaviour, including the same error messages :frowning:

7

Wow. I feel so incredibly stupid. Thanks for asking this very basic question, this problem was driving me nuts for 3 days straight, and I just never applied the configuration… Its not working yet, but I suspect thats an error on the configuration. I’ll report back once its fixed.

Edit: It is working now, I was stumbling across this bug in addition: [BUG] SAML endpoint still using _opendistro instead of _plugins · Issue #836 · opensearch-project/security-dashboards-plugin · GitHub

Login is working now. Thanks for all your help and the pointer to the rather simple solution.

8

@Anthony I’m thinking to put together a full end-to-end How-To for connecting OpenSearch with Keycloak as I’ve learned quite a bit in the previous days and also stumbled across a lot of forum posts and issues were people have similar pain. Haven’t seen any How-To section in particular, do you think its something that can or should be done here in forum?

9

@Wasabi Yes, totally, Community always appreciates knowledge sharing.
I would recommend perhaps to create a new threat/case and reference it here.

10

@Wasabi , did you ever put together a guide for OpenSearch and Keycloak?
I would be very interested in seeing that.