Configure SSO with SAML on Keycloak for Opensearch

Security
,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch and dashboards are on 2.18, Ubuntu 22.04 LTS, Keycloak - Version 26.0.8

Describe the issue:

I want to setup SingleSign on to Opensearch with keycloak and SAML, however when I click on “Log in with single-sign on” browser got redirected to https://opensearch.example.com:5601/auth/saml/login?redirectHash=false and I get an error message: “{“statusCode”:500,“error”:“Internal Server Error”,“message”:“Internal Error”}”

Configuration:

This is my /etc/opensearch/opensearch-security/config.yml:

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    # Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
    #filtered_alias_mode: warn
    #do_not_fail_on_forbidden: false
    #kibana:
    # Kibana multitenancy
    #multitenancy_enabled: true
    #private_tenant_enabled: true
    #default_tenant: ""
    #server_username: kibanaserver
    #index: '.kibana'
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        #remoteIpHeader:  'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      saml_auth_domain:
        order: 1
        description: "SAML provider"
        http_enabled: true
        transport_enabled: false
        http_authenticator:
          type: "saml"
          challenge: true
          config:
            idp:
              metadata_url: "https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor"
              entity_id: "keycloak.example.com:8443/realms/opensearch/"
            sp:
              entity_id: "https://opensearch.example.com:5601"
            kibana_url: "https://opensearch.example.com:5601"
            roles_key: "Role"
            exchange_key: "SECRET_KEY"
        authentication_backend:
          type: noop

I suspect that I missed something in my keycloak config, I added a realm opensearch and tried to add a client like this:

image
image1384×679 44.9 KB

Please not that I replaced my companys name with example.com in the config here, otherwise this is my current config.

I already tried to apply Pablos suggestions from Configuration for SAML login in OpenSearch with Keycloak but to no avail.

Any ideas how to troubleshoot this or where to look for some documentation/tutorial how to do this?

Best regards, Johannes.

Relevant Logs or Screenshots:

1 Like
2

@johannes_s Are you using a self-signed certificate in your Keycloak?
Could you check the OpenSearch logs and look for an SSL connection to IDP?

3

Kind of: It’s created with a smallstep ca, the generated certificates have a complete verifiable toolchain consisting of the keycloak certificate, our smallstep ca and the root ca of our company. The same kind of certificates are used for

I did a grep keycloak.example.com on the opensearch-example.log (again replaced companyname with bogus name otherwise log entries are unaltered):

opensearch-example.log:[2025-02-07T13:43:25,179][ERROR][o.o.s.m.r.i.HTTPMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
opensearch-example.log:[2025-02-07T13:43:25,180][ERROR][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error occurred while attempting to refresh metadata from 'https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor'
opensearch-example.log:net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor
opensearch-example.log:[2025-02-07T13:43:25,184][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Next refresh cycle for metadata provider 'https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor' will occur on '2025-02-07T12:44:25.180057988Z' ('2025-02-07T13:44:25.180057988+01:00[Europe/Berlin]' local time)
opensearch-example.log:Caused by: net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor
opensearch-example.log:[2025-02-07T13:43:26,285][ERROR][o.o.s.m.r.i.HTTPMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
opensearch-example.log:[2025-02-07T13:43:26,285][ERROR][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Error occurred while attempting to refresh metadata from 'https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor'
opensearch-example.log:net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor
opensearch-example.log:[2025-02-07T13:43:26,290][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [bfa-p-osm1.bghw.de] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Next refresh cycle for metadata provider 'https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor' will occur on '2025-02-07T12:44:26.285637138Z' ('2025-02-07T13:44:26.285637138+01:00[Europe/Berlin]' local time)
opensearch-example.log:Caused by: net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/
opensearch-example.log:com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.example.com:8443/realms/opensearch/

Do you need more loglines from that context? Does this mean that I need to add the smallstep ca cert file to opensearch TLS/SSL configuration?

Thanks for the support

4

@johannes_s This is your issue

Error retrieving metadata from https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The security plugin in the OpenSearch node fails to connect with Keycloak and retrieve metadata file.

The issue is related to the SSL certificate configured for your Keycloak’s HTTPS endpoint.
Any certificate that an external CA does not sign is self-signed. Your company’s CA is self-signed as it is not validated by anything else.
You have a few options here:

  1. Use HTTP endpoint instead
  2. Download descriptor file from Keycloack and place it in /usr/share/opensearch/config (https://keycloak.example.com:8443/realms/opensearch/protocol/saml/descriptor) and use metadata_file instead
        http_authenticator:
          type: "saml"
          challenge: true
          config:
            idp:
                metadata_file: descriptor
  1. Import your company’s CA or Keycloak’s certificate to your Linux machine’s keystore.
1 Like
5

Thanks for the hints and sorry for the late answer. I was busy with more urgent tasks before I could try the options. I used the option with the metadata_file (tbh not quite happy with it since it’s another manual step after a change in the keycloak config. But that’s another problem I’ll deal with later), now I get this error in the browser window after clicking “Log in with single-sign-on” on the login page of Opensearch Dashboards:

{“statusCode”:500,“error”:“Internal Server Error”,“message”:“Internal Error”}

I tried looking for 500 in the keycloak and opensearch logs but didn’t find anything interesting. I also found following messages in the opensearch-server.log but they might be unrelated (since we also have nodes writing data into the opensearch cluster and some of them might still be missconfigured):

Exception during establishing a SSL connection: java.net.SocketException: Connection reset
java.net.SocketException: Connection reset

Any ideas where to start looking in the logs? Would it be worth a try to create certificates directly in our organisations CA without the intermediate CA from Smallstep? I want to avoid to spend time with a red herring.

Regards, Johannes.

6

@johannes_s

Hey I had that issue a while back, perhaps this my help.

You may have to scroll down further on that post

1 Like
7

@johannes_s Could you share your opensearch_dashboards.yml file?

I understand that you’re not getting redirected to the IDP.

1 Like
8

Thanks @Gsmitt for your suggestion, sadly it still doesn’t work (see below)
@pablo Thanks again for the support. I again anonymized my companys name otherwise this is the current dashboards configuration:

---
# Copyright OpenSearch Contributors
# SPDX-License-Identifier: Apache-2.0

# Description:
# Default configuration for OpenSearch Dashboards

# OpenSearch Dashboards is served by a back end server. This setting specifies the port to use.
# server.port: 5601

# Specifies the address to which the OpenSearch Dashboards server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
# server.host: "localhost"
server.host: "opensearch-master.companyname.com"

# Enables you to specify a path to mount OpenSearch Dashboards at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell OpenSearch Dashboards if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
# server.basePath: ""

# Specifies whether OpenSearch Dashboards should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
# server.maxPayloadBytes: 1048576

# The OpenSearch Dashboards server's name.  This is used for display purposes.
# server.name: "your-hostname"
server.name: "opensearch-master.companyname.com"

# The URLs of the OpenSearch instances to use for all your queries.
# opensearch.hosts: ["http://localhost:9200"]

# OpenSearch Dashboards uses an index in OpenSearch to store saved searches, visualizations and
# dashboards. OpenSearch Dashboards creates a new index if the index doesn't already exist.
# opensearchDashboards.index: ".opensearch_dashboards"

# The default application to load.
# opensearchDashboards.defaultAppId: "home"

# Setting for an optimized healthcheck that only uses the local OpenSearch node to do Dashboards healthcheck.
# This settings should be used for large clusters or for clusters with ingest heavy nodes.
# It allows Dashboards to only healthcheck using the local OpenSearch node rather than fan out requests across all nodes.
#
# It requires the user to create an OpenSearch node attribute with the same name as the value used in the setting
# This node attribute should assign all nodes of the same cluster an integer value that increments with each new cluster that is spun up
# e.g. in opensearch.yml file you would set the value to a setting using node.attr.cluster_id:
# Should only be enabled if there is a corresponding node attribute created in your OpenSearch config that matches the value here
# opensearch.optimizedHealthcheckId: "cluster_id"

# If your OpenSearch is protected with basic authentication, these settings provide
# the username and password that the OpenSearch Dashboards server uses to perform maintenance on the OpenSearch Dashboards
# index at startup. Your OpenSearch Dashboards users still need to authenticate with OpenSearch, which
# is proxied through the OpenSearch Dashboards server.
# opensearch.username: "opensearch_dashboards_system"
# opensearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the OpenSearch Dashboards server to the browser.
# server.ssl.enabled: false
# server.ssl.certificate: /path/to/your/server.crt
# server.ssl.key: /path/to/your/server.key

server.ssl.enabled: true
server.ssl.certificate: /etc/opensearch/TLS/chain.pem
server.ssl.key: /etc/opensearch/TLS/key.pem


opensearch.ssl.alwaysPresentCertificate: true
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of OpenSearch Dashboards to OpenSearch and are required when
# xpack.security.http.ssl.client_authentication in OpenSearch is set to required.
# opensearch.ssl.certificate: /path/to/your/client.crt
opensearch.ssl.certificate: /etc/opensearch/TLS/chain.pem
# opensearch.ssl.key: /path/to/your/client.key
opensearch.ssl.key: /etc/opensearch/TLS/key.pem

# opensearch.ssl.certificate: /path/to/your/client.crt
# opensearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your OpenSearch instance.
# opensearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
opensearch.ssl.certificateAuthorities: [  "/usr/local/share/ca-certificates/company-root-ca-2021.crt", "/etc/opensearch/TLS/smallstep_ca.intermediate_ca.nobundle.crt", "/etc/opensearch/TLS/chain.pem"]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
# opensearch.ssl.verificationMode: full
opensearch.ssl.verificationMode: full


# Time in milliseconds to wait for OpenSearch to respond to pings. Defaults to the value of
# the opensearch.requestTimeout setting.
# opensearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or OpenSearch. This value
# must be a positive integer.
# opensearch.requestTimeout: 30000

# List of OpenSearch Dashboards client-side headers to send to OpenSearch. To send *no* client-side
# headers, set this value to [] (an empty list).
# opensearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to OpenSearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the opensearch.requestHeadersWhitelist configuration.
# opensearch.customHeaders: {}

# Time in milliseconds for OpenSearch to wait for responses from shards. Set to 0 to disable.
# opensearch.shardTimeout: 30000

# Logs queries sent to OpenSearch. Requires logging.verbose set to true.
# opensearch.logQueries: false

# Specifies the path where OpenSearch Dashboards creates the process ID file.
# pid.file: /var/run/opensearchDashboards.pid

# Enables you to specify a file where OpenSearch Dashboards stores log output.
# logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
# logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
# logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
# logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
# ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
# i18n.locale: "en"

# Set the allowlist to check input graphite Url. Allowlist is the default check list.
# vis_type_timeline.graphiteAllowedUrls: ['https://www.hostedgraphite.com/UID/ACCESS_KEY/graphite']

# Set the blocklist to check input graphite Url. Blocklist is an IP list.
# Below is an example for reference
# vis_type_timeline.graphiteBlockedIPs: [
#  //Loopback
#  '127.0.0.0/8',
#  '::1/128',
#  //Link-local Address for IPv6
#  'fe80::/10',
#  //Private IP address for IPv4
#  '10.0.0.0/8',
#  '172.16.0.0/12',
#  '192.168.0.0/16',
#  //Unique local address (ULA)
#  'fc00::/7',
#  //Reserved IP address
#  '0.0.0.0/8',
#  '100.64.0.0/10',
#  '192.0.0.0/24',
#  '192.0.2.0/24',
#  '198.18.0.0/15',
#  '192.88.99.0/24',
#  '198.51.100.0/24',
#  '203.0.113.0/24',
#  '224.0.0.0/4',
#  '240.0.0.0/4',
#  '255.255.255.255/32',
#  '::/128',
#  '2001:db8::/32',
#  'ff00::/8',
# ]
# vis_type_timeline.graphiteBlockedIPs: []

# opensearchDashboards.branding:
#   logo:
#     defaultUrl: ""
#     darkModeUrl: ""
#   mark:
#     defaultUrl: ""
#     darkModeUrl: ""
#   loadingLogo:
#     defaultUrl: ""
#     darkModeUrl: ""
#   faviconUrl: ""
#   applicationTitle: ""

# Set the value of this setting to true to capture region blocked warnings and errors
# for your map rendering services.
# map.showRegionBlockedWarning: false%

# Set the value of this setting to false to suppress search usage telemetry
# for reducing the load of OpenSearch cluster.
# data.search.usageTelemetry.enabled: false

# 2.4 renames 'wizard.enabled: false' to 'vis_builder.enabled: false'
# Set the value of this setting to false to disable VisBuilder
# functionality in Visualization.
# vis_builder.enabled: false

# 2.4 New Experimental Feature
# Set the value of this setting to true to enable the experimental multiple data source
# support feature. Use with caution.
# data_source.enabled: false
# Set the value of these settings to customize crypto materials to encryption saved credentials
# in data sources.
# data_source.encryption.wrappingKeyName: 'changeme'
# data_source.encryption.wrappingKeyNamespace: 'changeme'
# data_source.encryption.wrappingKey: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

# 2.6 New ML Commons Dashboards Feature
# Set the value of this setting to true to enable the ml commons dashboards
# ml_commons_dashboards.enabled: false

# 2.12 New experimental Assistant Dashboards Feature
# Set the value of this setting to true to enable the assistant dashboards
# assistant.chat.enabled: false

# 2.13 New Query Assistant Feature
# Set the value of this setting to false to disable the query assistant
# observability.query_assist.enabled: false

# 2.14 Enable Ui Metric Collectors in Usage Collector
# Set the value of this setting to true to enable UI Metric collections
# usageCollection.uiMetric.enabled: false

#opensearch.hosts: [https://localhost:9200]
opensearch.hosts: [https://opensearch-master.companyname.com:9200]
#opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: PASSWORD
opensearch.requestHeadersWhitelist: [authorization, securitytenant]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]
# Use this setting if you are running opensearch-dashboards without https
#opensearch_security.cookie.secure: false
opensearch_security.cookie.secure: true

# settings for saml/keycloak 
# https://opensearch.org/docs/latest/security/authentication-backends/saml#opensearch-dashboards-configuration
opensearch_security.auth.multiple_auth_enabled: true
opensearch_security.auth.type: [ "basicauth", "saml" ]
#opensearch_security.auth.type: "saml"
server.xsrf.allowlist: ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]
#opensearch_security.saml.extra_storage.cookie_prefix: security_authentication_saml
#opensearch_security.saml.extra_storage.additional_cookies: 3
## https://forum.opensearch.org/t/configuration-for-saml-login-in-opensearch-with-keycloak/20883/5
#opensearch.requestHeadersAllowlist: ["securitytenant","Authorization"]

I also tried the solution from @Gsmitt old thread now I’m getting this error message:

[2025-03-12T11:54:20,502][ERROR][c.a.d.a.h.s.HTTPSamlAuthenticator] [opensearch-master.companyname.com] Error in reRequestAuthentication()
com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml/descriptor
	at com.amazon.dlic.auth.http.saml.Saml2SettingsProvider.get(Saml2SettingsProvider.java:76) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.saml.Saml2SettingsProvider.getCached(Saml2SettingsProvider.java:118) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.reRequestAuthentication(HTTPSamlAuthenticator.java:204) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:305) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.filter.SecurityRestFilter.checkAndAuthenticateRequest(SecurityRestFilter.java:308) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:91) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:38) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1503) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1366) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1415) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
[2025-03-12T11:54:20,504][ERROR][c.a.d.a.h.s.HTTPSamlAuthenticator] [opensearch-server.companyname.com] Error in reRequestAuthentication()
com.amazon.dlic.auth.http.saml.SamlConfigException: Could not find entity descriptor for https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml/descriptor
	at com.amazon.dlic.auth.http.saml.Saml2SettingsProvider.get(Saml2SettingsProvider.java:76) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.saml.Saml2SettingsProvider.getCached(Saml2SettingsProvider.java:118) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.reRequestAuthentication(HTTPSamlAuthenticator.java:204) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:411) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.filter.SecurityRestFilter.checkAndAuthenticateRequest(SecurityRestFilter.java:308) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:91) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:38) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1503) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1366) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1415) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
[2025-03-12T11:54:20,505][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-server.companyname.com] Authentication finally failed for null from 10.30.179.53:37236

I’m using a metadata xml file. Any ideas?

9

Can you access this URL? Even when you use a descriptor file instead of the URL, OpenSearch and OpenSearch Dashboards must be able to resolve the name of your Keycloak VM?

2 Likes
10

Yes, this is how I generated the metadata file:

curl https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml/descriptor  |xmlindent | tee /tmp/descriptor.yml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3554  100  3554    0     0  54269      0 --:--:-- --:--:-- --:--:-- 54676
<md:EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" entityID="https://keycloak.companyname.com:8443/realms/opensearch">
    <md:IDPSSODescriptor WantAuthnRequestsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
        <md:KeyDescriptor use="signing">
            <ds:KeyInfo>
                <ds:KeyName>V4Mjxd5Ah4VFj_yvJ9GMLoyoWEk8XtXYtTNg_pk1SrU</ds:KeyName>
                <ds:X509Data>
                    <ds:X509Certificate>MIICozCCAYsCBgGSe5ABUDANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDDApvcGVuc2VhcmNoMB4XDTI0MTAxMTEyMjk0OFoXDTM0MTAxMTEyMzEyOFowFTETMBEGA1UEAwwKb3BlbnNlYXJjaDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL6HgLjzzo1x5tXMl2V3dqD1jopLx8+y6fSlrs8DE3vcmmEffkC1heJvPEn4MuDS7f7ugOGLj+on3GaltjK4HKbJIYcVh4BP/k90jLJb/rPFfBPvSjEze+1OkTT5ZGXG7Bug5VjiGfIWd7Rd6Q+qL8BHHIcQaqIdScuYl5G5Qs+pRj34q69F+S+wxemhGRh9kxEUwUpHMgIOtd8j0Y+fnIvE/S7VMf4euUvkmhLBQEXaHMRtHhLKsOWigQK+gF+JA8qVJcPet+L01v8PjZqm+tAqyJEIGl0O4zp0RGHhnxqYqOXVfs0RYMOV9DE/Imo0e1y0LliS4kJ8ox2nIuinm60CAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAQj5Et1GLXq2c50JZQ8jCGix9gGudlSBYZWFd1MhdJMFyuONiUsTNiWe9Et1NWX3EIUmslBU3fpPQjZbszRLjU4XxFqVRok6iDqTCPGO1xLZF9CIefourRi01/jPrJT9xfYljW35j0od6k3HFjhCzdudvL9oikCXB9hPJNTyDmNrnBBk20CyPpacr4kExKZHILPXuoT2ST/RjnMLgcSx75yaXNNevTLx3ofAPX8J7FfxJOXO9TcZzXk0Y6/NflblkM0lef3yrkrRO+kHlszQBFdxiNrKz6wFPwFmuWAfLstY9yZ5aZ5DYIXShuL7zTZXhrvp3UnvrLyQvSD5Ougve/Q==</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        <md:ArtifactResolutionService Binding="urn:oasis:names:tc:SAML:2.0:bindings:SOAP" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml/resolve" index="0"></md:ArtifactResolutionService><md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleLogoutService><md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleLogoutService><md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Artifact" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleLogoutService><md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:SOAP" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleLogoutService><md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:persistent</md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:transient</md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleSignOnService><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleSignOnService><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:SOAP" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleSignOnService><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Artifact" Location="https://keycloak.companyname.com:8443/realms/opensearch/protocol/saml"></md:SingleSignOnService>
    </md:IDPSSODescriptor>
</md:EntityDescriptor>

On the other hand following curl calls led to 404 errors, although they are referenced in the descriptor.xml file:

curl https://keycloak.companyname.com:8443/realms/opensearch/protocol/
{"error":"HTTP 404 Not Found"}

curl https://keycloak.companyname.com:8443/realms/opensearch/protocol/resolve
{“error”:“HTTP 404 Not Found”}


So my guess is, that I missed something important during the configuration of keycloak or opensearch. Any idea?
1 Like
11

@johannes_s Where did you run this curl command? How did you deploy your cluster? Is it docker or regular virtual machine?

I’m not aware of this “resolve” endpoint. 404 error could be correct for this link.

1 Like