Okta/OpenID(OIDC) authentication

Hi
the Openserarch version 2.0.1, I set as the document:OpenID Connect - OpenSearch documentation.
I visit the opensearch home page , it redirect to https://jp.test.com:5601/auth/openid/login?
and get the error:
{“statusCode”:401,“error”:“Unauthorized”,“message”:“Unauthorized”}

/etc/opensearch/opensearch.yml

# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]
node.max_local_storage_nodes: 3

/etc/opensearch/opensearch-security/config.yml

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    # Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
    #filtered_alias_mode: warn
    #do_not_fail_on_forbidden: false
    #kibana:
    # Kibana multitenancy
    #multitenancy_enabled: true
    #server_username: kibanaserver
    #index: '.kibana'
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        #remoteIpHeader:  'x-forwarded-for'
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            enable_ssl: true
            verify_hostnames: false
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: okta url/oauth2/default/.well-known/oauth-authorization-server
        authentication_backend:
          type: noop
      ldap:
        description: "Authenticate via LDAP or Active Directory"
        http_enabled: false
        transport_enabled: false
        order: 5
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          # LDAP authentication backend (authenticate users against a LDAP or Active Directory)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - localhost:8389
            bind_dn: null
            password: null
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(sAMAccountName={0})'
            # Use this attribute from the user as username (if not set then DN is used)
            username_attribute: null
    authz:
      roles_from_myldap:
        description: "Authorize via LDAP or Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - localhost:8389
            bind_dn: null
            password: null
            rolebase: 'ou=groups,dc=example,dc=com'
            # Filter to search for roles (currently in the whole subtree beneath rolebase)
            # {0} is substituted with the DN of the user
            # {1} is substituted with the username
            # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
            rolesearch: '(member={0})'
            # Specify the name of the attribute which value should be substituted with {2} above
            userroleattribute: null
            # Roles as an attribute of the user entry
            userrolename: disabled
            #userrolename: memberOf
            # The attribute in a role entry containing the name of that role, Default is "name".
            # Can also be "dn" to use the full DN as rolename.
            rolename: cn
            # Resolve nested roles transitive (roles which are members of other roles and so on ...)
            resolve_nested_roles: true
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(uid={0})'
            # Skip users matching a user name, a wildcard or a regex pattern
            #skip_users:
            #  - 'cn=Michael Jackson,ou*people,o=TEST'
            #  - '/\S*/'
      roles_from_another_ldap:
        description: "Authorize via another Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: ldap
          #config goes here ...
  #    auth_failure_listeners:
  #      ip_rate_limiting:
  #        type: ip
  #        allowed_tries: 10
  #        time_window_seconds: 3600
  #        block_expiry_seconds: 600
  #        max_blocked_clients: 100000
  #        max_tracked_clients: 100000
  #      internal_authentication_backend_limiting:
  #        type: username
  #        authentication_backend: intern
  #        allowed_tries: 10
  #        time_window_seconds: 3600
  #        block_expiry_seconds: 600
  #        max_blocked_clients: 100000
  #        max_tracked_clients: 100000

/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

logging.verbose: true

server.port: 5601
server.host: "0.0.0.0"
opensearch.hosts: [https://localhost:9200]
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersAllowlist: ["Authorization", "security_tenant"]
opensearch.ssl.certificateAuthorities: ["/etc/opensearch/root-ca.pem"]

server.ssl.enabled: true
server.ssl.certificate: /usr/share/opensearch-dashboards/config/jp.test.com.crt
server.ssl.key: /usr/share/opensearch-dashboards/config/jp.test.com.key

opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "okta url/oauth2/default/.well-known/oauth-authorization-server"
opensearch_security.openid.client_id: "0oa123456789R6697"
opensearch_security.openid.client_secret: "zCafdafdafdsfafsxcvxvdsdonf_dfadsaf-_V"
opensearch_security.openid.scope: "openid profile email"
opensearch_security.openid.header: "Authorization"
opensearch_security.openid.base_redirect_url: "opensearch url/auth/openid/login"

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]
# Use this setting if you are running opensearch-dashboards without https
opensearch_security.cookie.secure: true

Thanks

1 Like

@gehf The opensearch_security.openid.base_redirect_url: must point your Opensearch Dashboards URL (i.e. https://<OSD_IP_FQDN>:5601 or https://<OSD_IP_FQDN> if proxied).

The authentication order should be:

  1. basic auth
  2. openid

The openid URL should end “oauth2/default/.well-known/openid-configuration”
Have you checked your OKTA URL in the browser?
Have you checked OpenSearch logs? Do you see any errors in regards to OKTA connection?

@pablo thank for you reply.
I change the `opensearch_security.openid.base_redirect_url: like: https://jp.test.com:5601.
and basic auth order is 0, openid order is 1.
the opensearch_security.openid.connect_url: also change like https://test.okta.com/oauth2/default/.well-known/openid-configuration.
this url can access in brower, the result like:

{
	"issuer": "https://test.okta.com/oauth2/default",
	"authorization_endpoint": "https://test.okta.com/oauth2/default/v1/authorize",
	"token_endpoint": "https://test.okta.com/oauth2/default/v1/token",
	"userinfo_endpoint": "https://test.okta.com/oauth2/default/v1/userinfo",
	"registration_endpoint": "https://test.okta.com/oauth2/v1/clients",
	"jwks_uri": "https://test.okta.com/oauth2/default/v1/keys",
	"response_types_supported": ["code", "id_token", "code id_token", "code token", "id_token token", "code id_token token"],
	"response_modes_supported": ["query", "fragment", "form_post", "okta_post_message"],
	"grant_types_supported": ["authorization_code", "implicit", "refresh_token", "password", "urn:ietf:params:oauth:grant-type:device_code"],
	"subject_types_supported": ["public"],
	"id_token_signing_alg_values_supported": ["RS256"],
	"scopes_supported": ["openid", "profile", "email", "address", "phone", "offline_access", "device_sso"],
	"token_endpoint_auth_methods_supported": ["client_secret_basic", "client_secret_post", "client_secret_jwt", "private_key_jwt", "none"],
	"claims_supported": ["iss", "ver", "sub", "aud", "iat", "exp", "jti", "auth_time", "amr", "idp", "nonce", "name", "nickname", "preferred_username", "given_name", "middle_name", "family_name", "email", "email_verified", "profile", "zoneinfo", "locale", "address", "phone_number", "picture", "website", "gender", "birthdate", "updated_at", "at_hash", "c_hash"],
	"code_challenge_methods_supported": ["S256"],
	"introspection_endpoint": "https://test.okta.com/oauth2/default/v1/introspect",
	"introspection_endpoint_auth_methods_supported": ["client_secret_basic", "client_secret_post", "client_secret_jwt", "private_key_jwt", "none"],
	"revocation_endpoint": "https://test.okta.com/oauth2/default/v1/revoke",
	"revocation_endpoint_auth_methods_supported": ["client_secret_basic", "client_secret_post", "client_secret_jwt", "private_key_jwt", "none"],
	"end_session_endpoint": "https://test.okta.com/oauth2/default/v1/logout",
	"request_parameter_supported": true,
	"request_object_signing_alg_values_supported": ["HS256", "HS384", "HS512", "RS256", "RS384", "RS512", "ES256", "ES384", "ES512"],
	"device_authorization_endpoint": "https://test.okta.com/oauth2/default/v1/device/authorize",
	"pushed_authorization_request_endpoint": "https://test.okta.com/oauth2/default/v1/par"
}

I try it again, the same error:
{“statusCode”:401,“error”:“Unauthorized”,“message”:“Unauthorized”}
the opensearch log:

 log   [12:40:49.628] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [12:40:49.627]  GET / 302 3ms - 9.0B
  log   [12:40:49.644] [debug][server][OpenSearchDashboards][cookie-session-storage][http] Error: Unauthorized
respons [12:40:49.644]  GET /auth/openid/login 401 2ms - 9.0B
  log   [12:40:52.702] [debug][metrics] Refreshing metrics

thanks.

Hengfeng

@pablo I try reset our customer’s environment as you told, it can redirect to okta, after login Okta, it will redirect to the opensearch openid login home page like: https://jp.test.com:5601/auth/openid/login?, then can’t login successful, jump back to okta, then redirect many times and failed. it seems the user’s roles not in the parameter post by okta. Do I need to add this test user in the internal_users.yml? and add roles/roles_maps for this user?

thanks.

@gehf Could you share your Claims from the Okta?

@pablo
it is default Authorization Servers in Okta:


the default Claims:

and the Token Preview like this:
{
  "sub": "00dsafafadaffadfdafazxcz",
  "name": "Hengfeng Ge",
  "email": "gehf@test.com",
  "ver": 1,
  "iss": "https://test.okta.com/oauth2/default",
  "aud": "afdasfafdafdafdasfaf",
  "iat": 1612323851,
  "exp": 1623243251,
  "jti": "ID.afdasfazvczx_qrezzdafdaavzadfadaf",
  "amr": [
    "pwd"
  ],
  "idp": "00oadfaeqezvcafapl696",
  "nonce": "nonce",
  "preferred_username": "gehf",
  "auth_time": 1000,
  "at_hash": "preview_at_hash"
}

thanks.

@gehf Your JWT token doesn’t contain roles. This is caused by the missing roles claim in OKTA.
That’s why your authentication succeeds and authorization fails.

Try this instead.

@pablo thanks for your reply.
we add the roles claim as you told. it also in the okta preview token.

we have map the everyone roles to all_access. it still redirect many times.
Could you check if there are any other suggestions?

thanks.

@pablo
the setting:

these setting still not working.

thanks.

@pablo Hello Pablo, Could you check these issue when you available? it seems the roles setting has been configured.

thanks.

@gehf The backend role must be assigned to the role instead of the internal user. That’s how the roles mapping works in the security plugin.

image

@gehf Could you share your roles_mapping.yml file?

@pablo thanks. the user with opensearch app is in the group: security_group in Okta.
we want map this okta group: security_group to opensearch backend_roles: all_access.
the test user is in security_group in Okta. in the admin console, security_group map with all_access.


the token information:

the roles_mapping:

---
# In this file users, backendroles and hosts can be mapped to Security roles.
# Permissions for OpenSearch roles are configured in roles.yml

_meta:
  type: "rolesmapping"
  config_version: 2

# Define your roles mapping here

## Demo roles mapping

all_access:
  reserved: false
  backend_roles:
  - "admin"
  - "security_group"
  description: "Maps admin to all_access"

own_index:
  reserved: false
  users:
  - "*"
  description: "Allow full access to an index named like the username"

logstash:
  reserved: false
  backend_roles:
  - "logstash"

kibana_user:
  reserved: false
  backend_roles:
  - "kibanauser"
  description: "Maps kibanauser to kibana_user"

readall:
  reserved: false
  backend_roles:
  - "readall"

manage_snapshots:
  reserved: false
  backend_roles:
  - "snapshotrestore"

kibana_server:
  reserved: true
  users:
  - "kibanaserver"

the okta group: security_group is in yml for all_access, but the test result as same as before.

we find the error message:


in the OIDC guide, Default is Authorization. why it still get the error No ‘Basic Authorization’ header, send 401 and ‘www-Authenticate Basic’.

thanks.

@gehf Have you changed the order in config.yml? I see that your openid has the order 1 now, but don’t know if you changed the basicauth to 0.

Did you upload this config to the cluster when you modified the order?

Your all_access mapping should work now.

@pablo yes, the order of basicauth has been changed to 0.

 authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            enable_ssl: true
            verify_hostnames: false
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://test.okta.com/oauth2/default/.well-known/openid-configuration
        authentication_backend:
          type: noop

but it stil redirect many times then failed.
is it No ‘Basic Authorization’ header issue? is there any setting we should confirm?

thanks.

@gehf The mentioned error could be caused by kibanaserver user as OpenSearch Dashboards will try to authenticate against OpenSearch with both basicauth and openid authentications.
Try adding skip_users options as per the below example.

      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            enable_ssl: true
            verify_hostnames: false
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://test.okta.com/oauth2/default/.well-known/openid-configuration
            skip_users:
              - kibanaserver
        authentication_backend:
          type: noop

Could you share login section of your application in the Okta?

@pablo we add the skip_users as this:


and the okta login page:

the error message as before, redirect many times, then failed.

thanks.

@gehf I’ve tried to repro your issue but can’t. I even tried with Ubuntu desktop and localhost as OpenSearch Desktop redirect.

Did you use securityadmin.sh script to upload your config changes?

@pablo yes, has runned securityadmin.sh after config change. still meet the redirect issue.
I meet another issue like this:

Is it possible that we have a meeting with okta support together? If possible, please give me an email or IM ID, I will contact you. thanks for you help.

@gehf Could you check your OpenSearch Dashboards and OpenSearch nodes TLS certificates? I had something similar when I had my certs expired.