Getting statusCode 404 with SAML integration

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

2.15

Describe the issue:

Getting statusCode 404

Jul 8 15:29:30 opensearch-server opensearch-dashboards[27963]: {“type”:“response”,“@timestamp”:“2024-07-08T15:29:30Z”,“tags”:,“pid”:27963,“method”:“post”,“statusCode”:404,“req”:{“url”:“/_plugins/_security/saml/acs”,“method”:“post”,“headers”:{“host”:“opensearch-server.mfm.pvt:5601”,“connection”:“keep-alive”,“content-length”:“10093”,“cache-control”:“max-age=0”,“sec-ch-ua”:“"Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"”,“sec-ch-ua-mobile”:“?0”,“sec-ch-ua-platform”:“"Windows"”,“upgrade-insecure-requests”:“1”,“origin”:https://mpgus.okta.com,“content-type”:“application/x-www-form-urlencoded”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“cross-site”,“sec-fetch-mode”:“navigate”,“sec-fetch-dest”:“document”,“referer”:https://mpgus.okta.com/,“accept-encoding”:“gzip, deflate, br, zstd”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“192.168.152.149”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”,“referer”:https://mpgus.okta.com/},“res”:{“statusCode”:404,“responseTime”:8,“contentLength”:9},“message”:“POST /_plugins/_security/saml/acs 404 8ms - 9.0B”}

Configuration:

Relevant Logs or Screenshots:

@egraham Please share the content of opensearch_dashboards.yml and config.yml files.
What SAML IdP do you use?

opensearch-dashboards.yml


server.host: "opensearch-server.mfm.pvt"

opensearch.hosts: [[https://127.0.0.1:9200](https://127.0.0.1:9200/), [https://opensearch-server:9200](https://opensearch-server:9200/), [https://opensearch-server.mfm.pvt:9200](https://opensearch-server.mfm.pvt:9200/)]

server.ssl.enabled: true

opensearch.ssl.verificationMode: none

opensearch.username: kibanaserver

opensearch.password: kibanaserver

opensearch.requestHeadersWhitelist: [authorization, securitytenant]

server.ssl.certificate: /usr/share/opensearch-dashboards/admin.pem

server.ssl.key: /usr/share/opensearch-dashboards/admin.key

opensearch.ssl.certificateAuthorities: [ "/usr/share/opensearch-dashboards/MPG-CA.pem"]

opensearch_security.multitenancy.enabled: true

opensearch_security.multitenancy.tenants.preferred: [Private, Global]

opensearch_security.readonly_mode.roles: [kibana_read_only]

# Use this setting if you are running opensearch-dashboards without https

opensearch_security.cookie.secure: true

#opensearch_security.auth.type: "saml"

#server.xsrf.allowlist: ["/_plugins/_security/saml/acs/idpinitiated", "/_plugins/_security/saml/acs", "/_plugins/_security/saml/logout", "/_opendistro/_security/saml/acs"]

config.yml

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    # Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
    #filtered_alias_mode: warn
    #do_not_fail_on_forbidden: false
    #kibana:
    # Kibana multitenancy
    #multitenancy_enabled: true
    #private_tenant_enabled: true
    #default_tenant: ""
    #server_username: kibanaserver
    #index: '.kibana'
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        #remoteIpHeader:  'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
#      saml_auth_domain:
#        http_enabled: true
#        transport_enabled: false
#        order: 1
#       http_authenticator:
#          type: saml
#          challenge: true
#          config:
#            idp:
#              metadata_url: https://REDACTED.okta.com/app/REDACTED/sso/saml/metadata
#              entity_id: https://REDACTED.okta.com/app/REDACTED_opensearchdashboard_1/REDACTED
#            sp:
#              entity_id: opensearch-saml
#            kibana_url: "REDACTED"
#            roles_key: roles
#            exchange_key: "REDACTED"
#        authentication_backend:
#          type: noop
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
    authz:
      roles_from_myldap:
        description: "Authorize via LDAP or Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - localhost:8389
            bind_dn: null
            password: null
            rolebase: 'ou=groups,dc=example,dc=com'
            # Filter to search for roles (currently in the whole subtree beneath rolebase)
            # {0} is substituted with the DN of the user
            # {1} is substituted with the username
            # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
            rolesearch: '(member={0})'
            # Specify the name of the attribute which value should be substituted with {2} above
            userroleattribute: null
            # Roles as an attribute of the user entry
            userrolename: disabled
            #userrolename: memberOf
            # The attribute in a role entry containing the name of that role, Default is "name".
            # Can also be "dn" to use the full DN as rolename.
            rolename: cn
            # Resolve nested roles transitive (roles which are members of other roles and so on ...)
            resolve_nested_roles: true
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(uid={0})'
            # Skip users matching a user name, a wildcard or a regex pattern
            #skip_users:
            #  - 'cn=Michael Jackson,ou*people,o=TEST'
            #  - '/\S*/'
      roles_from_another_ldap:
        description: "Authorize via another Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: ldap
          #config goes here ...

@egraham Please use the following entry in OpenSearch Dashboards as per documentation.

server.xsrf.allowlist: ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]

You should change _plugins/_security/saml/acs to _opendistro/_security/saml/acs in the OKTA client.

I believe we already tried this but will try again. Thank you.

@egraham My conclusion regarding OKTA is based on the provided error.

statusCode”:404,“responseTime”:8,“contentLength”:9},“message”:“POST /_plugins/_security/saml/acs 404 8ms - 9.0B

_plugins/_security/saml/acs is an invalid endpoint and it never worked in Opensearch.

I made that change but am still getting a 404 error.

Jul 12 11:12:03 opensearch-server opensearch-dashboards[851]: {“type”:“response”,“@timestamp”:“2024-07-12T11:12:03Z”,“tags”:,“pid”:851,“method”:“get”,“statusCode”:401,“req”:{“url”:“/_opendistro/_security/saml/acs”,“method”:“get”,“headers”:{“host”:“opensearch-server.mfm.pvt:5601”,“connection”:“keep-alive”,“sec-ch-ua”:“"Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"”,“sec-ch-ua-mobile”:“?0”,“sec-ch-ua-platform”:“"Windows"”,“upgrade-insecure-requests”:“1”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“none”,“sec-fetch-mode”:“navigate”,“sec-fetch-user”:“?1”,“sec-fetch-dest”:“document”,“accept-encoding”:“gzip, deflate, br, zstd”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“192.168.152.149”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”},“res”:{“statusCode”:401,“responseTime”:2,“contentLength”:9},“message”:“GET /_opendistro/_security/saml/acs 401 2ms - 9.0B”}

Jul 12 11:12:03 opensearch-server opensearch-dashboards[851]: {“type”:“response”,“@timestamp”:“2024-07-12T11:12:03Z”,“tags”:,“pid”:851,“method”:“get”,“statusCode”:401,“req”:{“url”:“/favicon.ico”,“method”:“get”,“headers”:{“host”:“opensearch-server.mfm.pvt:5601”,“connection”:“keep-alive”,“sec-ch-ua”:“"Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"”,“sec-ch-ua-mobile”:“?0”,“user-agent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”,“sec-ch-ua-platform”:“"Windows"”,“accept”:“image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8”,“sec-fetch-site”:“same-origin”,“sec-fetch-mode”:“no-cors”,“sec-fetch-dest”:“image”,“referer”:https://opensearch-server.mfm.pvt:5601/_opendistro/_security/saml/acs,“accept-encoding”:“gzip, deflate, br, zstd”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“192.168.152.149”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36”,“referer”:https://opensearch-server.mfm.pvt:5601/_opendistro/_security/saml/acs},“res”:{“statusCode”:401,“responseTime”:1,“contentLength”:9},“message”:“GET /favicon.ico 401 1ms - 9.0B”}

@pablo - Thanks for your help so far. Any thoughts?

@egraham What response do you get when you try to access the below link through the browser?

https://opensearch-server.mfm.pvt:5601/_opendistro/_security/saml/acs

@egraham I’ve just noticed that you’re getting a different error - HTTP 401 Unauthorized.

Now it is unauthorized. This is a progress.

How did you configure your roles in OKTA?

In Opensearch UI under roles. I have okta_dashboards_admin mapped as a backend role to the all_access role. I have okta_dashboards_user mapped as a backend role to the kibana_user role.

In OKTA, I have the following:

@egraham I assume that you’ve assigned OKTA users to OKTA groups.
You could try increasing logging on SAML in OpenSearch logs and investigate the SAML token content sent by OKTA after successful authentication.

I was testing mine with the following setting.
image

But mine is more global than yours.