Unable to configure SAML with Azure AD

[anubisg1]

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch 2.4.0
OpenSearch dashboards 2.4.0
OpenSearch kubernetes operator 2.1.1

Describe the issue:

Can’t make SAML work,

Access from Azure test (idpinitiated) returns:
{"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

Direct access from OpenSearch Dashboards returns:
{"statusCode":500,"error":"Internal Server Error","message":"Internal Error"}

Configuration:

Not sure if relevant, but this is all in kuberntes, and the dashboard is behing nginx ingress with the following config

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: opensearch-dashboard-ingress
  namespace: opensearch-dev
  annotations:
    nginx.ingress.kubernetes.io/session-cookie-samesite: "None"
    nginx.ingress.kubernetes.io/session-cookie-path: "/; Secure"
    nginx.ingress.kubernetes.io/session-cookie-conditional-samesite-none: "true"   
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
    nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
    nginx.ingress.kubernetes.io/proxy-body-size: 8m
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - opensearch-dev.xxxxxxx.com
      secretName: ingress-xxxxx-com
  rules:
    - host: opensearch-dev.xxxxxxxx.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: opensearch-cluster-dashboards
              port:
                number: 5601

dashboards config

      opensearch_security.auth.multiple_auth_enabled: "true"
      opensearch_security.auth.type: |
        ["basicauth","saml"]
      server.xsrf.allowlist: |
        ["/_plugins/_security/saml/acs/idpinitiated", "/_plugins/_security/saml/acs", "/_plugins/_security/saml/logout"]
      # Multi Tenancy
      opensearch.requestHeadersAllowlist: |
        ["securitytenant","Authorization"]

config.yaml

      config.yml: |-
        _meta:
          type: "config"
          config_version: "2"
        config:
          dynamic:
            do_not_fail_on_forbidden: true
            http:
              anonymous_auth_enabled: true                                
            authc:
              basic_internal_auth_domain:
                http_enabled: true
                transport_enabled: false
                order: 0
                http_authenticator:
                  type: basic
                  challenge: false
                authentication_backend:
                  type: internal
              saml_auth_domain:
                http_enabled: true
                transport_enabled: false
                order: 1
                http_authenticator:
                  type: saml
                  challenge: true
                  config:
                    idp:
                      metadata_file: https://login.microsoftonline.com/xxxxxxxxx/federationmetadata/2007-06/federationmetadata.xml?appid=xxxxxxxxxxxxxx
                      entity_id: https://sts.windows.net/xxxxxxxxxxxx/
                    sp:
                      entity_id: https://opensearch-dev.xxxxxxx.com/
                    kibana_url: https://opensearch-dev.xxxxxxxxx.com/
                    roles_key: http://schemas.microsoft.com/ws/2008/06/identity/claims/role
                    exchange_key: 'xxxxxxxxxx'
                authentication_backend:
                  type: noop

Relevant Logs or Screenshots:

logs from dashboards:

{"type":"response","@timestamp":"2022-12-01T09:28:27Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/_plugins/_security/saml/idpinitiated","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"625dd72fef6efcd7e1e8757d26f78c6c","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"cross-site","sec-fetch-mode":"navigate","sec-fetch-dest":"document","referer":"https://account.activedirectory.windowsazure.com/","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://account.activedirectory.windowsazure.com/"},"res":{"statusCode":401,"responseTime":1,"contentLength":9},"message":"GET /_plugins/_security/saml/idpinitiated 401 1ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:28:28Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"12ca5903dbf8a597844fecc3da282614","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"https://opensearch-dev.xxxxxxxx.com/_plugins/_security/saml/idpinitiated","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/_plugins/_security/saml/idpinitiated"},"res":{"statusCode":401,"responseTime":2,"contentLength":9},"message":"GET /favicon.ico 401 2ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/api/v1/restapiinfo","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"31c5a1bb8c6217ea4af9d159cd28d1f3","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /api/v1/restapiinfo 401 3ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/api/v1/configuration/account","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"6e80db8da3b23e247cd3dafe3b9b24d7","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":401,"responseTime":1,"contentLength":9},"message":"GET /api/v1/configuration/account 401 1ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"post","statusCode":200,"req":{"url":"/api/core/capabilities","method":"post","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"9b86c8360a4df82f465b05b4276a98ac","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","content-length":"527","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","origin":"https://opensearch-dev.xxxxxxxx.com","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":200,"responseTime":4,"contentLength":9},"message":"POST /api/core/capabilities 200 4ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/api/v1/auth/type","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"458923b54763b9842abad3c6c794effe","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":401,"responseTime":2,"contentLength":9},"message":"GET /api/v1/auth/type 401 2ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/api/v1/multitenancy/tenant","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"77445ead4b0af5dbb7b263404ebcb839","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":401,"responseTime":2,"contentLength":9},"message":"GET /api/v1/multitenancy/tenant 401 2ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:10Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/api/v1/configuration/account","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"6584fa9d95c399fe8a3f6eafce832c99","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","content-type":"application/json","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","osd-version":"2.4.0","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/app/login?"},"res":{"statusCode":401,"responseTime":1,"contentLength":9},"message":"GET /api/v1/configuration/account 401 1ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:12Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/api/reporting/stats","method":"get","headers":{"host":"10.244.4.222:5601","user-agent":"kube-probe/1.24","accept":"*/*","connection":"close"},"remoteAddress":"10.244.4.1","userAgent":"kube-probe/1.24"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"GET /api/reporting/stats 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:13Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/auth/saml/captureUrlFragment?nextUrl=%2F","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"c379206e93fb90d1cc849fcc14934fba","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"GET /auth/saml/captureUrlFragment?nextUrl=%2F 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:13Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/auth/saml/captureUrlFragment.js","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"1f0799624b2051d3a150cc1863ca2aeb","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"script","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/captureUrlFragment?nextUrl=%2F","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/captureUrlFragment?nextUrl=%2F"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /auth/saml/captureUrlFragment.js 200 3ms - 9.0B"}
{"type":"log","@timestamp":"2022-12-01T09:29:14Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"Failed to get saml header: Error: Invalid SAML configuration."}
{"type":"error","@timestamp":"2022-12-01T09:29:13Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:143:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:97:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:92:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:164:34)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"http://opensearch-dev.xxxxxxxx.com/auth/saml/login?nextUrl=%2F&redirectHash=false","message":"Internal Server Error"}
{"type":"response","@timestamp":"2022-12-01T09:29:13Z","tags":[],"pid":1,"method":"get","statusCode":500,"req":{"url":"/auth/saml/login?nextUrl=%2F&redirectHash=false","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"3f991108ccee512edb4315f8042d6a58","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","sec-fetch-dest":"document","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/captureUrlFragment?nextUrl=%2F","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/captureUrlFragment?nextUrl=%2F"},"res":{"statusCode":500,"responseTime":214,"contentLength":9},"message":"GET /auth/saml/login?nextUrl=%2F&redirectHash=false 500 214ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T09:29:14Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"opensearch-dev.xxxxxxxx.com","x-request-id":"9b3628c5605c611964d76853df237bf5","x-real-ip":"172.19.136.7","x-forwarded-for":"172.19.136.7","x-forwarded-host":"opensearch-dev.xxxxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/login?nextUrl=%2F&redirectHash=false","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxxxx.com/auth/saml/login?nextUrl=%2F&redirectHash=false"},"res":{"statusCode":401,"responseTime":2,"contentLength":9},"message":"GET /favicon.ico 401 2ms - 9.0B"}

[pablo]

@anubisg1 Values of server.xsrf.allowlist are incorrect. Please follow the documentation.

SAML

Double-check your exchange key. It must be at least 32 characters.

SAML

I’m not sure if roles_key value will work for you. Try the below instead.

roles_key: "http://schemas.microsoft.com/ws/2008/06/identity/claims/groups"

Just be aware that roles from Azure will appear as Group IDs.

[anubisg1]

Hello pablo,
why do you say that " server.xsrf.allowlist are incorrect " ? yes, looking at the saml configuration instructions it says that it should start with " /_opendistro/ "

but the troubleshooting guide as well the bug below:

Troubleshoot SAML

[BUG] SAML endpoint still using _opendistro instead of _plugins

the endpoint changed from /_opendistro/ to /_plugins/ therefore it seems to be a miss in the documentation… i can add them both of course, but i need to know which one is the correct one as i need to configure that on the azure side as well

the exchange_key right now is a 64chars long string

i will try a different roles_key as you suggest the one i use is based on the following two links

Azure sso with saml issue

Hello, I believe your issue is in roles_key. Since you are using azure app registration as IDP try add http://schemas.microsoft.com/ws/2008/06/identity/claims/role as values to roles_key and re-deploy to see if that works. Dont forget to run securityadmin.sh The issue basically could be that no roles are fetch from the SAML assertion , if you debug the SAML request you can see what is sent from azure. Please redeploy opensearch and opensearch-dashboards.

The Role of Claims

thanks for your feedback, i’ll go try

BTW, i’m running version 2.4.0, you?

[pablo]

@anubisg1 Have a look at this SAML auth uses legacy `_opendistro` route · Issue #2060 · opensearch-project/security · GitHub.

OS is still using the legacy endpoint for SAML.

The Troubleshooting section is incorrect and will be changed.

[anubisg1]

Thanks…

so, this is how i changed my config right now (still no success though)

opensearch_dashboards.yml

opensearch.requestHeadersAllowlist: ["securitytenant","Authorization"]
opensearch_security.auth.type: saml
server.maxPayloadBytes: 8388608
server.name: opensearch-cluster-dashboards
server.xsrf.allowlist: ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]

config.yml

_meta:
  type: "config"
  config_version: "2"
config:
  dynamic:
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: true
    authc:
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      saml_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: saml
          challenge: true
          config:
            idp:
              metadata_file: https://login.microsoftonline.com/xxxxxx/federationmetadata/2007-06/federationmetadata.xml?appid=xxxxxx
              entity_id: https://sts.windows.net/xxxxx/
            sp:
              entity_id: https://opensearch-dev.xxxx.com/
            kibana_url: https://opensearch-dev.xxxxx.com/
            roles_key: http://schemas.microsoft.com/ws/2008/06/identity/claims/groups
            exchange_key: '7b7de37274960c88148211472d3975a8'
        authentication_backend:
          type: noop

Azure config

i still get the same erros though, and while it claims that Failed to get saml header: Error: Invalid SAML configuration i really don’ know what else can be wrong now

{"type":"response","@timestamp":"2022-12-01T13:57:55Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"opensearch-dev.xxxxxx.com","x-request-id":"f40b547d046e59e1c09f0b41d3e30bee","x-real-ip":"10.244.4.1","x-forwarded-for":"10.244.4.1","x-forwarded-host":"opensearch-dev.xxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"},"res":{"statusCode":302,"responseTime":4,"contentLength":9},"message":"GET / 302 4ms - 9.0B"}
{"type":"log","@timestamp":"2022-12-01T13:57:55Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"Failed to get saml header: Error: Invalid SAML configuration."}
{"type":"error","@timestamp":"2022-12-01T13:57:55Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:143:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:97:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:92:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:164:34)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"http://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","message":"Internal Server Error"}
{"type":"response","@timestamp":"2022-12-01T13:57:55Z","tags":[],"pid":1,"method":"get","statusCode":500,"req":{"url":"/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","method":"get","headers":{"host":"opensearch-dev.xxxxxx.com","x-request-id":"8abb8853651a74af659d175cadc35ce1","x-real-ip":"10.244.4.1","x-forwarded-for":"10.244.4.1","x-forwarded-host":"opensearch-dev.xxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"},"res":{"statusCode":500,"responseTime":47,"contentLength":9},"message":"GET /auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards 500 47ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T13:57:55Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"opensearch-dev.xxxxxx.com","x-request-id":"54d9ec62716f374080b76926d0ed01e0","x-real-ip":"10.244.4.1","x-forwarded-for":"10.244.4.1","x-forwarded-host":"opensearch-dev.xxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"https://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","referer":"https://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards"},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /favicon.ico 401 3ms - 9.0B"}
{"type":"response","@timestamp":"2022-12-01T13:58:04Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"opensearch-dev.xxxxxx.com","x-request-id":"236bb42e9fce4c7245b73b2c739b13ea","x-real-ip":"10.244.4.1","x-forwarded-for":"10.244.4.1","x-forwarded-host":"opensearch-dev.xxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","purpose":"prefetch","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET / 302 2ms - 9.0B"}
{"type":"log","@timestamp":"2022-12-01T13:58:04Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"Failed to get saml header: Error: Invalid SAML configuration."}
{"type":"error","@timestamp":"2022-12-01T13:58:04Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:143:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:97:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:92:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:164:34)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"http://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","message":"Internal Server Error"}
{"type":"response","@timestamp":"2022-12-01T13:58:04Z","tags":[],"pid":1,"method":"get","statusCode":500,"req":{"url":"/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards","method":"get","headers":{"host":"opensearch-dev.xxxxxx.com","x-request-id":"a46e09d69e084f5b6b48d407221dffa0","x-real-ip":"10.244.4.1","x-forwarded-for":"10.244.4.1","x-forwarded-host":"opensearch-dev.xxxxxx.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-scheme":"https","x-scheme":"https","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","purpose":"prefetch","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","sec-ch-ua":"\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,it;q=0.8,cs;q=0.7"},"remoteAddress":"10.244.4.11","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"},"res":{"statusCode":500,"responseTime":8,"contentLength":9},"message":"GET /auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards 500 8ms - 9.0B"}

[pablo]

@anubisg1 In your configs you use the below URL for OpenSearch Dashboards.

https://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards

However, the Internal Server Error refers to

http://opensearch-dev.xxxxxx.com/auth/saml/login?nextUrl=%2Fapp%2Fopensearch-dashboards

When you get the Internal Server Error in the browser do you see HTTP or HTTPS?

[anubisg1]

@pablo

u get the Internal Server Error in the browser do you see HTTP or HTTPS?

all “external access” is via https because there is NGINX reverse proxy in front of dashboards doing ssl offload (with valid certificates). from nginx onwards is http.

opensearch and opensearch dashboards are reachable inside kubernetes via http://opensearch-cluster-dashboards:5601 then kubernetes ingress exposes https://opensearch-dev.xxxxxx.com to the external world (users and azure) …

i tried to configure kibana_url: http://opensearch-cluster-dashboards:5601 but that didn’t seem to change anything

other 2 things i can try to do is.

kibana_url: http://opensearch-dev.xxxx.com/

or enable https on opensearch dashboards and enable nginx.ingress.kubernetes.io/force-ssl-redirect: "true" on nginx ingress

[anubisg1]

i tried both and didn’t change much…

that http vs https miss match you saw is not longer there, but i still get the same errors

[pablo]

@anubisg1 I run my test deployment as docker-compose. I’ll try to test it with Nginx.

[pablo]

@anubisg1 Just noticed in your config.yml.

Use

metadata_url

instead of

metadata_file

I’ve tested SAML authentication with haproxy and it worked.

[anubisg1]

@pablo thank you for the time you spent and thank you for catching that as well…

i changed it but i still get the same exact error Failed to get saml header: Error: Invalid SAML configuration

i’m guess that i will need to enable log4j debug and look a bit deeper…

[anubisg1]

While i still can’t get SAML to work, i was able to get openid to work right away so i will be using this…

i would still like to try and get SAML working , if possible