Opensearch Dashboard OpenID not working

I’m trying to setup OpenID authentication in Opensearch Dashboards with Azure Active Directory and I’ve hit a wall.

If I don’t have the openid configs in opensearch_dashboards.yml, a login screen comes up and I can log into Opensearch Dashboards with an internal user without issue. However, if I attempt to enable openid in Opensearch Dashboards, attempting to open the Opensearch Dashboards URL will just result in a 401 error page with the following content:
{"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

I’ve got the following for the configs in opensearch_dashboards.yml for openid(identifying/secret information removed of course):

opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/###TENANT ID###/v2.0/.well-known/openid-configuration"
opensearch_security.openid.scope: "openid"
opensearch_security.openid.client_id: "###Azure App ID###"
opensearch_security.openid.client_secret: "###Azure App Secret###"

Just in case, I’ve already tried creating and using a new secret, but the same error comes up. Not sure where to look from here to continue troubleshooting.

@thozook Could you share your config.yml and opensearch-dashboards.yml files?

Here’s my opensearch_dashboards.yml

---
# Copyright OpenSearch Contributors
# SPDX-License-Identifier: Apache-2.0

# Description:
# Default configuration for OpenSearch Dashboards

# OpenSearch Dashboards is served by a back end server. This setting specifies the port to use.
# server.port: 5601

# Specifies the address to which the OpenSearch Dashboards server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "##DASHBOARDS HOSTNAME##"

# Enables you to specify a path to mount OpenSearch Dashboards at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell OpenSearch Dashboards if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
# server.basePath: ""

# Specifies whether OpenSearch Dashboards should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# server.rewriteBasePath: false

# The maximum payload size in bytes for incoming server requests.
# server.maxPayloadBytes: 1048576

# The OpenSearch Dashboards server's name.  This is used for display purposes.
server.name: "Opensearch Dashboards"

# The URLs of the OpenSearch instances to use for all your queries.
opensearch.hosts: ["https://##OPENSEARCH HOSTNAME##:9200"]
# OpenSearch Dashboards uses an index in OpenSearch to store saved searches, visualizations and
# dashboards. OpenSearch Dashboards creates a new index if the index doesn't already exist.
# opensearchDashboards.index: ".opensearch_dashboards"

# The default application to load.
# opensearchDashboards.defaultAppId: "home"

# Setting for an optimized healthcheck that only uses the local OpenSearch node to do Dashboards healthcheck.
# This settings should be used for large clusters or for clusters with ingest heavy nodes.
# It allows Dashboards to only healthcheck using the local OpenSearch node rather than fan out requests across all nodes.
#
# It requires the user to create an OpenSearch node attribute with the same name as the value used in the setting
# This node attribute should assign all nodes of the same cluster an integer value that increments with each new cluster that is spun up
# e.g. in opensearch.yml file you would set the value to a setting using node.attr.cluster_id:
# Should only be enabled if there is a corresponding node attribute created in your OpenSearch config that matches the value here
# opensearch.optimizedHealthcheckId: "cluster_id"

# If your OpenSearch is protected with basic authentication, these settings provide
# the username and password that the OpenSearch Dashboards server uses to perform maintenance on the OpenSearch Dashboards
# index at startup. Your OpenSearch Dashboards users still need to authenticate with OpenSearch, which
# is proxied through the OpenSearch Dashboards server.
opensearch.username: "kibanaserver"
opensearch.password: "kibanaserver"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the OpenSearch Dashboards server to the browser.
server.ssl.enabled: true
server.ssl.certificate: /etc/opensearch-dashboards/certs/##TLS CERT FILENAME##
server.ssl.key: /etc/opensearch-dashboards/certs/##TLS KEY FILENAME##

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of OpenSearch Dashboards to OpenSearch and are required when
# xpack.security.http.ssl.client_authentication in OpenSearch is set to required.
# opensearch.ssl.certificate: /path/to/your/client.crt
# opensearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your OpenSearch instance.
opensearch.ssl.certificateAuthorities: [ "/etc/opensearch-dashboards/certs/CACertBundle.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
opensearch.ssl.verificationMode: certificate

# Time in milliseconds to wait for OpenSearch to respond to pings. Defaults to the value of
# the opensearch.requestTimeout setting.
# opensearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or OpenSearch. This value
# must be a positive integer.
# opensearch.requestTimeout: 30000

# List of OpenSearch Dashboards client-side headers to send to OpenSearch. To send *no* client-side
# headers, set this value to [] (an empty list).
opensearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to OpenSearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the opensearch.requestHeadersWhitelist configuration.
# opensearch.customHeaders: {}

# Time in milliseconds for OpenSearch to wait for responses from shards. Set to 0 to disable.
# opensearch.shardTimeout: 30000

# Logs queries sent to OpenSearch. Requires logging.verbose set to true.
# opensearch.logQueries: false

# Specifies the path where OpenSearch Dashboards creates the process ID file.
pid.file: /var/run/opensearch-dashboards/opensearchDashboards.pid

# Enables you to specify a file where OpenSearch Dashboards stores log output.
logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
# ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
# i18n.locale: "en"

# Set the allowlist to check input graphite Url. Allowlist is the default check list.
# vis_type_timeline.graphiteAllowedUrls: ['https://www.hostedgraphite.com/UID/ACCESS_KEY/graphite']

# Set the blocklist to check input graphite Url. Blocklist is an IP list.
# Below is an example for reference
# vis_type_timeline.graphiteBlockedIPs: [
#  //Loopback
#  '127.0.0.0/8',
#  '::1/128',
#  //Link-local Address for IPv6
#  'fe80::/10',
#  //Private IP address for IPv4
#  '10.0.0.0/8',
#  '172.16.0.0/12',
#  '192.168.0.0/16',
#  //Unique local address (ULA)
#  'fc00::/7',
#  //Reserved IP address
#  '0.0.0.0/8',
#  '100.64.0.0/10',
#  '192.0.0.0/24',
#  '192.0.2.0/24',
#  '198.18.0.0/15',
#  '192.88.99.0/24',
#  '198.51.100.0/24',
#  '203.0.113.0/24',
#  '224.0.0.0/4',
#  '240.0.0.0/4',
#  '255.255.255.255/32',
#  '::/128',
#  '2001:db8::/32',
#  'ff00::/8',
# ]
# vis_type_timeline.graphiteBlockedIPs: []

# opensearchDashboards.branding:
#   logo:
#     defaultUrl: ""
#     darkModeUrl: ""
#   mark:
#     defaultUrl: ""
#     darkModeUrl: ""
#   loadingLogo:
#     defaultUrl: ""
#     darkModeUrl: ""
#   faviconUrl: ""
#   applicationTitle: ""

# Set the value of this setting to true to capture region blocked warnings and errors
# for your map rendering services.
# map.showRegionBlockedWarning: false%

# Set Auth Type
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/##TENANT ID##/v2.0/.well-known/openid-configuration"
opensearch_security.openid.scope: "openid"
opensearch_security.openid.client_id: "##APP ID##"
opensearch_security.openid.client_secret: "##APP SECRET##"

# Set Data Directory
path.data: /var/lib/opensearch-dashboards/

and here’s the config.yml:

---

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      openid_auth_domain:
        description: "OpenID connection to Azure AD"
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: email
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/##TENANT ID##/v2.0/.well-known/openid-configuration
            openid_connect_idp:
              enable_ssl: true
              verify_hostnames: true
              pemtrustedcas_filepath: /etc/opensearch/certs/ca-certificates.crt
        authentication_backend:
          type: noop
    authz:

@thozook Can you try the below instead?

      openid_auth_domain:
        description: "OpenID connection to Azure AD"
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: name
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/##TENANT ID##/v2.0/.well-known/openid-configuration
        authentication_backend:
          type: noop

In opensearch-dashboards.yml

  1. Add securitytenant to requestHeadersWhitelist
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
  1. Remove
opensearch_security.openid.scope: "openid"

@pablo Tried these changes, but Dashboards gave the same 401 error message.

@thozook I don’t see opensearch_security.openid.base_redirect_url:

Try setting it with opensearch_security.openid.base_redirect_url: "https://opensearch-dashboards:5601"

Do you use a reverse proxy to access opensearch-dashboards?
In Azure, your redirect URL should be https://opensearch-dashboards:5601/auth/openid/login

You can also follow this troubleshooting guide and check the content of JWT returned by Azure.

Do you use private mode in the browser to test this solution? If not, try to clear cookies.

@pablo I’ve set opensearch_security.openid.base_redirect_url: "https://##DASHBOARDS HOSTNAME##:5601" in opensearch_dashboards.yml and restarted dashboards, but no change in the error.

Opensearch Dashboards is not being run behind a reverse proxy and the redirect URL in Azure looks correct:

I am using Private Mode in my browser when I access Opensearch Dashboards(closing the Private Window after every test to ensure subsequent tests aren’t affect by previous ones).

@thozook Do you get to the Microsoft login page or the error appears before that?

@pablo The error appears before that. As soon as I open the Dashboards URL, I get redirected to /auth/openid/login?nextUrl=%2F and the error appears. This only happens when the OpenID configs are in opensearch_dashboards.yml. If I remove them or comment them out, everything works with the internal users(login page appears and I can login to Dashboards).

An update: I stopped Opensearch Dashboards and cleared out the uuid file in my data path(/var/lib/opensearch-dashboards) and now the authentication is working. Is there some interaction between OpenID Connect and the value in that file?