OpenSearch in Kubernetes: OpenID error

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch 2.6.0, kubernetes, all browsers

Describe the issue:
Hello!

We use Wazuh as our SIEM, it is built on top of OpenSearch. The installation is running in a Kubernetes cluster.

We would like to implement OpenID Connect to the OpenSearch Dashboard.

We get, like many others, “Access denied” after the successful authentication to the IDP.

We have enabled the “plugins.security.allow_default_init_securityindex” to pick up the new configuration in the cluster.

Now, we have “Basic”, and “OpenID Connect” authentication methods enabled, with basic we can log in. I don’t think however that it applies the proper configuration, as it shows the old, original entries in the security settings under “Authentication sequences” below.

Could somebody help us with this please?

Thanks,
Andras

Configuration:
opensearch_dashboard.yml:

server.host: 0.0.0.0
server.port: 5601
opensearch.hosts: h_t_t_p_s://indexer:9200
opensearch.ssl.verificationMode: none
opensearch.requestHeadersWhitelist: [ authorization,securitytenant,Authorization ]
opensearch_security.multitenancy.enabled: false
opensearch_security.readonly_mode.roles: [“kibana_read_only”]
server.ssl.enabled: true
server.ssl.key: “/usr/share/wazuh-dashboard/certs/key.pem”
server.ssl.certificate: “/usr/share/wazuh-dashboard/certs/cert.pem”
opensearch.ssl.certificateAuthorities: [“/usr/share/wazuh-dashboard/certs/root-ca.pem”]
uiSettings.overrides.defaultRoute: /app/wazuh
opensearch_security.cookie.secure: true
opensearch_security.cookie.password: “RANDOM”

opensearch_security.auth.multiple_auth_enabled: true
# opensearch_security.auth.type: [“basicauth”,“openid”,“jwt”]
opensearch_security.auth.type: [“openid”,“basicauth”]
# opensearch_security.auth.type: [“jwt”,“basicauth”]
opensearch_security.ui.openid.login.buttonname: “Log in with COMPANY Account”
opensearch_security.openid.scope: “openid profile email offline_access”
opensearch_security.openid.trust_dynamic_headers: true
# opensearch_security.openid.verify_hostnames: false
#opensearch_security.openid.scope: openid profile email
*#AAD → *
opensearch_security.openid.connect_url: “h_t_t_p_s://login.microsoftonline.com/TENANTID/v2.0/.well-known/openid-configuration”
opensearch_security.openid.client_id: “CLIENTID”
opensearch_security.openid.client_secret: “CLIENTSECRET”
opensearch_security.openid.base_redirect_url: “h_t_t_p_s://HTTP_HOST:443”

# Some extra logging:
OpenSearch.logQueries: true
logging.verbose: true

config.yml:
_meta:
type: “config”
config_version: 2

config:
dynamic:
# Set filtered_alias_mode to ‘disallow’ to forbid more than 2 filtered aliases per index
# Set filtered_alias_mode to ‘warn’ to allow more than 2 filtered aliases per index but warns about it (default)
# Set filtered_alias_mode to ‘nowarn’ to allow more than 2 filtered aliases per index silently
#filtered_alias_mode: warn
#do_not_fail_on_forbidden: false
#kibana:
# Kibana multitenancy
#multitenancy_enabled: true
#server_username: kibanaserver
#index: ‘.kibana’
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: ‘192.168.0.10|192.168.0.11’ # regex pattern
authc:
basic_internal_auth_domain:
description: “Authenticate via HTTP Basic against internal users database”
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: basic
challenge: false
authentication_backend:
type: internal
openid_auth_domain:
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: openid
challenge: false
config:
openid_connect_idp:
enable_ssl: true
verify_hostnames: false
subject_key: preferred_username
roles_key: roles
openid_connect_url: h_t_t_p_s://login.microsoftonline.com/TENANTID/v2.0/.well-known/openid-configuration
authentication_backend:
type: noop
authz:

opensearch.yml:
cluster.name: ${CLUSTER_NAME}
node.name: ${NODE_NAME}
network.host: ${NETWORK_HOST}
discovery.seed_hosts: wazuh-indexer-0.wazuh-indexer
cluster.initial_master_nodes:

    • wazuh-indexer-0*

node.max_local_storage_nodes: “3”
path.data: /usr/share/wazuh-indexer/data
path.logs: /usr/share/wazuh-indexer/logs
plugins.security.ssl.http.pemcert_filepath: /usr/share/wazuh-indexer/certs/node.pem
plugins.security.ssl.http.pemkey_filepath: /usr/share/wazuh-indexer/certs/node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /usr/share/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.transport.pemcert_filepath: /usr/share/wazuh-indexer/certs/node.pem
plugins.security.ssl.transport.pemkey_filepath: /usr/share/wazuh-indexer/certs/node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /usr/share/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.authcz.admin_dn:

    • CN=admin,O=Company,L=California,C=US*
      plugins.security.check_snapshot_restore_write_privileges: true
      plugins.security.enable_snapshot_restore_privilege: true
      plugins.security.nodes_dn:
    • CN=.wazuh-indexer,O=Company,L=California,C=US
      plugins.security.restapi.roles_enabled:
      - “all_access”
      - “security_rest_api_access”
      plugins.security.allow_default_init_securityindex: true
      cluster.routing.allocation.disk.threshold_enabled: false
      compatibility.override_main_response_version: true
      http.max_header_size: 16k

Relevant Logs or Screenshots:
Authentication sequences:

image
image2998×1453 432 KB

Indexer logs:
[2023-07-19T12:08:51,922][WARN ][o.o.s.h.HTTPBasicAuthenticator] [wazuh-indexer-2] No ‘Basic Authorization’ header, send 401 and ‘WWW-Authenticate Basic’
[2023-07-19T12:08:51,922][WARN ][o.o.s.a.BackendRegistry ] [wazuh-indexer-2] No ‘Authorization’ header, send 401 and ‘WWW-Authenticate Basic’
Dashboard logs:
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:46Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:48Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:48Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:49Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:49Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:49Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:39,“message”:“Error: Unauthorized”}
{“type”:“response”,“@timestamp”:“2023-07-19T12:08:51Z”,“tags”:,“pid”:39,“method”:“get”,“statusCode”:302,“req”:{“url”:“/auth/openid/login”,“method”:“get”,“headers”:{“host”:“HTTP_HOST”,“connection”:“Keep-Alive”,“accept-encoding”:“gzip”,“x-forwarded-for”:“77.221.60.126”,“cf-ray”:“7e92da266ca40d63-VIE”,“x-forwarded-proto”:“https”,“cf-visitor”:“{"scheme":"https"}”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8”,“accept-language”:“en-US,en;q=0.5”,“upgrade-insecure-requests”:“1”,“sec-fetch-dest”:“document”,“sec-fetch-mode”:“navigate”,“sec-fetch-site”:“same-origin”,“sec-fetch-user”:“?1”,“cdn-loop”:“cloudflare”,“cf-connecting-ip”:“77.221.60.126”,“cf-ipcountry”:“HU”},“remoteAddress”:“10.0.10.35”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0”},“res”:{“statusCode”:302,“responseTime”:2,“contentLength”:9},“message”:“GET /auth/openid/login 302 2ms - 9.0B”}
{“type”:“log”,“@timestamp”:“2023-07-19T12:08:51Z”,“tags”:[“error”,“plugins”,“securityDashboards”],“pid”:39,“message”:“OpenId authentication failed: Error: Authentication Exception”}
{“type”:“response”,“@timestamp”:“2023-07-19T12:08:51Z”,“tags”:,“pid”:39,“method”:“get”,“statusCode”:401,“req”:{“url”:“/auth/openid/login?code=0.AVoAY_iBryrpjkuT9AEiFe_16bDKVaE0kfJDrLJjA2YpcypaAEk.AgABAAIAAAD–DLA3VO7QrddgJg7WevrAgDs_wUA9P-1ja1sT-7J4mitoASTT0rhEmWKNcxWmZg3eb74duK7mowsGEQfcxHDsICxwb4FL-0LCS8KkISmsahrVNQIqvk5XHxASLFBH3KPPMT2c7cpyo33xTiWq_XqkFdmoARGRlSXZFxxDm4AIWBLEUHHpxDq1ddXwNYkmlMrzycihQCx_ZuYaH8Bt2wB3lCeMvs1flNHqf8q37mKejJtgTMeD4zdXdC_k0UNqHDMCFvEKm-_KVY7h7MEqGYHgblWMhNGV_Zx2R_751_27Osi_HXpbJ4IPgIof-kwh3eA-EN9AzWGYOZCXuaaJZ2uehXJ-K_OC86cwM8Csp0xnLFcRp_PhST6aUMWLxqZlGJf_I3iw1EqmoQjIm6Fsc9xgxI7hWiBA-NlvMhX6Q9e71ZNGOTolw7tDPRle99_qpkt2ej0gL5c5haqE-KFu5SFizjvBybkmhrzBFSy8rpiQpHODCghjkQdjHqn8IsqKs6-08zaOKKwiKXFiFkM1lNG3yteARhifpNm2lAD0n8-uHprA0lEwkDtAlwbsBVRa1ACM-Rmls0n3QSVmAA4XEdR1QHQMhn4gUd7VNWU_MbH6c76sh8UtjqmrwZ9c8mHr41lMY9Bki0EIYp1T-KHe2UtfhbakHnmFUd4h5TduO1PaTVnqcN1fapEDtyEPnO5YrhpGIeU5mFwMNGqZgkzUGsxN2x7fBIih2iWA2xNCn_hQ5MJOZhHANRX5UM8XHk7dVpiKiOrfuRGLVrVE6rIqJ8nGx-yXFk&state=vJTwW3wwJZzIyQzsXIBGny&session_state=964a114a-2d04-4922-9b23-57ae288b9fd2”,“method”:“get”,“headers”:{“host”:“HTTP_HOST”,“connection”:“Keep-Alive”,“accept-encoding”:“gzip”,“x-forwarded-for”:“77.221.60.126”,“cf-ray”:“7e92da2a5e810d63-VIE”,“x-forwarded-proto”:“https”,“cf-visitor”:“{"scheme":"https"}”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8”,“accept-language”:“en-US,en;q=0.5”,“upgrade-insecure-requests”:“1”,“sec-fetch-dest”:“document”,“sec-fetch-mode”:“navigate”,“sec-fetch-site”:“none”,“sec-fetch-user”:“?1”,“cdn-loop”:“cloudflare”,“cf-connecting-ip”:“77.221.60.126”,“cf-ipcountry”:“HU”},“remoteAddress”:“10.0.10.35”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0”},“res”:{“statusCode”:401,“responseTime”:283,“contentLength”:9},“message”:“GET /auth/openid/login?code=0.AVoAY_iBryrpjkuT9AEiFe_16bDKVaE0kfJDrLJjA2YpcypaAEk.AgABAAIAAAD–DLA3VO7QrddgJg7WevrAgDs_wUA9P-1ja1sT-7J4mitoASTT0rhEmWKNcxWmZg3eb74duK7mowsGEQfcxHDsICxwb4FL-0LCS8KkISmsahrVNQIqvk5XHxASLFBH3KPPMT2c7cpyo33xTiWq_XqkFdmoARGRlSXZFxxDm4AIWBLEUHHpxDq1ddXwNYkmlMrzycihQCx_ZuYaH8Bt2wB3lCeMvs1flNHqf8q37mKejJtgTMeD4zdXdC_k0UNqHDMCFvEKm-_KVY7h7MEqGYHgblWMhNGV_Zx2R_751_27Osi_HXpbJ4IPgIof-kwh3eA-EN9AzWGYOZCXuaaJZ2uehXJ-K_OC86cwM8Csp0xnLFcRp_PhST6aUMWLxqZlGJf_I3iw1EqmoQjIm6Fsc9xgxI7hWiBA-NlvMhX6Q9e71ZNGOTolw7tDPRle99_qpkt2ej0gL5c5haqE-KFu5SFizjvBybkmhrzBFSy8rpiQpHODCghjkQdjHqn8IsqKs6-08zaOKKwiKXFiFkM1lNG3yteARhifpNm2lAD0n8-uHprA0lEwkDtAlwbsBVRa1ACM-Rmls0n3QSVmAA4XEdR1QHQMhn4gUd7VNWU_MbH6c76sh8UtjqmrwZ9c8mHr41lMY9Bki0EIYp1T-KHe2UtfhbakHnmFUd4h5TduO1PaTVnqcN1fapEDtyEPnO5YrhpGIeU5mFwMNGqZgkzUGsxN2x7fBIih2iWA2xNCn_hQ5MJOZhHANRX5UM8XHk7dVpiKiOrfuRGLVrVE6rIqJ8nGx-yXFk&state=vJTwW3wwJZzIyQzsXIBGny&session_state=964a114a-2d04-4922-9b23-57ae288b9fd2 401 283ms - 9.0B”}

2

@andraspavelbaystream According to the shared screenshot you didn’t enable OpenID in the OpenSearch plugin. As you can notice, only basic authentication is enabled.

Keep in mind that OpenSearch security plugin use .opendistro_security index to keep its configuration. Once the index is created, the only way to update it is with secruityadmin.sh script (recommended) or with API calls (not recommended) and admin certificates.

Did you use OpenSearch helm charts to deploy the cluster?