AWS IAM as IDP for SSO fails with a 500 Internal Server Error

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch version : 2.16.0
opensearch dasboard : 2.16.0
Browser : Chrome

Describe the issue:

I created an AWS IAM SAML application to configure SSO for our opensearch dashboard. Upon clicking on the app, it redirects to a 500 internal server error page as shown in the screenshots section. Upon inspecting the dashboard logs, the error seems to be a 401 unauthorized authentication exception. Regarding the security plugin saml section, I tried w/ and w/out subject_key, jwt_clock_skew_tolerance_seconds, pemtrustedcas_content, and exchange_key with no progress.

Regarding idp.pemtrustedcas_content, and when creating an app, a certificate is issued.
I first assumed that the certificate is publicly trusted and no need to add it but eventually added its content in idp.pemtrustedcas_content but no difference was noticed.

What I am trying to do is to map my AWS user group to a role. I have traced the saml request and the user groups are sent as expected under Role attribute. I mapped
Role->user.email instead of user.groups and mapped the email as a backend_role in role_mappings.yml with no results.

Error Two
I get another error when I click on the SSO button directly from the opensearch dashboard. I expect it to redirect to AWS for logging in but it also results in a 500 internal server error. However, opensearch dashboard logs says Error: failed parsing SAML config. Screenshots below!

I have been blocked on this for a few days and would appreciate any help :slight_smile:

Configuration:
Dashboard Configuration

dashboards:
      enable: true
      replicas: 1
      version: 2.16.0
      env:
      additionalConfig:
        opensearch_security.auth.type: |
          ["basicauth", "saml"]
        opensearch_security.auth.multiple_auth_enabled: "true"
        server.xsrf.allowlist: |
          ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]

Nodes(config.yml):

          authc:
            basic_internal_auth_domain:
              description: "Authenticate via HTTP Basic against internal users database"
              http_enabled: true
              transport_enabled: true
              order: 0
              http_authenticator:
                type: basic
                challenge: true
              authentication_backend:
                type: intern

            saml_auth_domain:
              http_enabled: true
              transport_enabled: false
              order: 1
              http_authenticator:
                type: saml
                challenge: true
                config:
                  idp:
                    metadata_url: https://portal.sso.eu-central-1.amazonaws.com/saml/metadata/<>
                    entity_id: https://portal.sso.eu-central-1.amazonaws.com/saml/assertion/<>
                    pemtrustedcas_content: |-
                      <AWS IAM App Certificate Content>
                  sp:
                    entity_id: https://opensearch-dashboard.tooling.<>.com
                  kibana_url: https://opensearch-dashboard.tooling.<>.com:5601/
                  roles_key: Role
                  subject_key: Subject
                  jwt_clock_skew_tolerance_seconds: 120
                  exchange_key: '9a2h8ajasdfhsdiydfn7dtd6d5ashsd89a2h8ajasdHhsdiyLfn7dtd6d5ashsdI'
              authentication_backend:
                type: noop

roles_mapping.yaml:

roles_mapping.yml: |-
        _meta:
          type: "rolesmapping"
          config_version: 2
        all_access:                 
          reserved: false
          backend_roles:
          - "admin"
          - "<Assigned AWS Group ID>"
          description: "Maps admin to all_access"

AWS App Metadata

Relevant Logs or Screenshots:
Dashboard Logs:

StatusCodeError: Authentication Exception
    at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
    at IncomingMessage.emit (node:events:529:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  status: 401,
  displayName: 'AuthenticationException',
  path: '/_plugins/_security/api/authtoken',
  query: { auth_type: 'saml' },
  body: 'Unauthorized',
  statusCode: 401,
  response: 'Unauthorized',
  wwwAuthenticateDirective: 'Basic realm="OpenSearch Security"',
  toString: [Function (anonymous)],
  toJSON: [Function (anonymous)],
  isBoom: true,
  isServer: false,
  data: null,
  output: {
    statusCode: 401,
    payload: {
      statusCode: 401,
      error: 'Unauthorized',
      message: 'Authentication Exception'
    },
    headers: { 'WWW-Authenticate': 'Basic realm="Authorization Required"' }
  },
  [Symbol(OpenSearchError)]: 'OpenSearch/notAuthorized'
}
{"type":"log","@timestamp":"2024-11-04T14:58:43Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"SAML IDP initiated authentication workflow failed: Error: failed to get token"}
{"type":"error","@timestamp":"2024-11-04T14:58:43Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:127:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:83:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:79:17)\n 

Error Two Dashboard Logs

Error: failed parsing SAML config                                                                                                                        │
│     at SecurityClient.getSamlHeader (/usr/share/opensearch-dashboards/plugins/securityDashboards/server/backend/opensearch_security_client.ts:214:15)
,"pid":1,"method":"get","statusCode":500,"req":{"url":"/auth/saml/login?redirectHash=false","method":"get","headers":{"host":"opensearch-dashboard.tooling.....

Hi @mostafa_sawy,

Any reason why you have challenge: true on both basic_internal_auth_domain and saml_auth_domain?

could you test with:

            basic_internal_auth_domain:
              description: "Authenticate via HTTP Basic against internal users database"
              http_enabled: true
              transport_enabled: true
              order: 0
              http_authenticator:
                type: basic
                challenge: false
              authentication_backend:
                type: intern

best,
mj

Hello @Mantas ,

Thanks for pointing that out, I must have missed it.

I did it try setting it to false now but same errors still occur.

SSO Button from the dashboard redirects to this(Error Two):

could you please share the output of the following:

curl --insecure -u <admin_username>:<admin_password> -XGET https://<OS_node>:9200/_plugins/_security/api/securityconfig?pretty

best,
mj

@Mantas I edited saml_auth_domain a bit from what I posted yesterday but same errors still persist

{
  "config" : {
    "dynamic" : {
      "filtered_alias_mode" : "warn",
      "disable_rest_auth" : false,
      "disable_intertransport_auth" : false,
      "respect_request_indices_options" : false,
      "kibana" : {
        "multitenancy_enabled" : true,
        "private_tenant_enabled" : true,
        "default_tenant" : "",
        "server_username" : "kibanaserver",
        "index" : ".kibana",
        "sign_in_options" : [
          "BASIC"
        ]
      },
      "http" : {
        "anonymous_auth_enabled" : false,
        "xff" : {
          "enabled" : false,
          "internalProxies" : "10\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}|192\\.168\\.\\d{1,3}\\.\\d{1,3}|169\\.254\\.\\d{1,3}\\.\\d{1,3}|127\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}|172\\.1[6-9]{1}\\.\\d{1,3}\\.\\d{1,3}|172\\.2[0-9]{1}\\.\\d{1,3}\\.\\d{1,3}|172\\.3[0-1]{1}\\.\\d{1,3}\\.\\d{1,3}",
          "remoteIpHeader" : "X-Forwarded-For"
        }
      },
      "authc" : {
        "basic_internal_auth_domain" : {
          "http_enabled" : true,
          "order" : 0,
          "http_authenticator" : {
            "challenge" : false,
            "type" : "basic",
            "config" : { }
          },
          "authentication_backend" : {
            "type" : "intern",
            "config" : { }
          },
          "description" : "Authenticate via HTTP Basic against internal users database"
        },
        "saml_auth_domain" : {
          "http_enabled" : true,
          "order" : 1,
          "http_authenticator" : {
            "challenge" : true,
            "type" : "saml",
            "config" : {
              "idp" : {
                "metadata_url" : "https://portal.sso.eu-central-1.amazonaws.com/saml/metadata/<>",
                "entity_id" : "https://portal.sso.eu-central-1.amazonaws.com/saml/assertion/<>"
              },
              "sp" : {
                "entity_id" : "https://opensearch-dashboard.tooling.<>.com"
              },
              "kibana_url" : "https://opensearch-dashboard.tooling.<>.com",
              "roles_key" : "Role",
              "subject_key" : "Subject"
            }
          },
          "authentication_backend" : {
            "type" : "noop",
            "config" : { }
          }
        }
      },
      "authz" : { },
      "auth_failure_listeners" : { },
      "do_not_fail_on_forbidden" : false,
      "multi_rolespan_enabled" : true,
      "hosts_resolver_mode" : "ip-only",
      "do_not_fail_on_forbidden_empty" : false,
      "on_behalf_of" : {
        "enabled" : false
      }
    }
  }
}

How did you deploy your cluster?

@Mantas Using the Opensearch k8s operator

Are you using Cognito for saml?

@Mantas Nope. I created an AWS IAM Identity Center SAML App

Hi @mostafa_sawy,

The “Hash=false” indicates that there is a good chance the “metadata_url” is not accessible.
(I have managed to successfully redirect the user to AWS using idp.metadata_url and idp.metadata_file).

Could you try running the below on your OS node container (let me know if you get an expected output):

curl -XGET https://portal.sso.us-east-1.amazonaws.com/saml/metadata/<YOUR_ID>

Or use the file, make sure the owner and group of the file are opensearch:opensearch and permissions of 755 (just for testing.)

best,
mj

You can check the SAML response. In my case, I’m using keycloak and the issue was how the Roles were serialized:

Before:

<saml:AttributeStatement>
            <saml:Attribute Name="Role"
                            NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"
                            >
                <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema"
                                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                     xsi:type="xs:string"
                                     >view-clients</saml:AttributeValue>
            </saml:Attribute>
            <saml:Attribute Name="Role"
                            NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"
                            >
                <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema"
                                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                     xsi:type="xs:string"
                                     >view-profile</saml:AttributeValue>
            </saml:Attribute>
           ...
</saml:AttributeStatement>

And I had to ensure it uses Single Role Attribute, to generate as follows (basically collate all values into a single attribute).


        <saml:AttributeStatement>
            <saml:Attribute Name="Role"
                            NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"
                            >
                <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema"
                                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                     xsi:type="xs:string"
                                     >view-clients</saml:AttributeValue>
                <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema"
                                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                     xsi:type="xs:string"
                                     >view-profile</saml:AttributeValue>
            </saml:Attribute>
        </saml:AttributeStatement>

This fixed the issue for me.