OIDC Integration: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

B3n · April 25, 2023, 5:45am

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch - 2.6.0
OpenSearch Dashboards - 2.6.0
docker-compose - 1.29.2
docker 20.10.17-ce

Describe the issue:
Hello together!

I’m currently trying to get the OIDC integration to work for the OpenSearch Dashboards. But so far i think i’m a bit lost in all the possibilities the config and documentation is offering to me.

So basicially all i want to do is using an OIDC provider, and for testing purposes i have it all made within localhost. So e.g. i’m using port forwarding to open “https://localhost:5601” to get into my instance, also i’ve configured that host within my OIDC provider as redirect URL (to be exact, it is: “https://localhost:5601/auth/openid/login”).

It almost seems to work as i at least come to the login screen to this OIDC provider, which then brings me back to “https://localhost:5601/auth/openid/login?code=xxx”, but with the message in the subject of this thread.

My guess is, that the information coming back from the provider do not match with the expected claims/scopes. But i can not really verify that, as i couldn’t find out how to log that.

Even:

logging.verbose: true

Just brought me the following:

opensearch-node2         | [2023-04-21T12:45:31,138][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node2] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-node2         | [2023-04-21T12:45:31,138][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-node2] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-dashboards    | {"type":"log","@timestamp":"2023-04-21T12:45:31Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"

I do have two valid users to login to my backend. The admin user and a user i’ve added to my internal users file.

Do i have to setup some kind matching between the internal user and the claims coming from the OIDC provider? If yes, where and how can i do that the best way? Also the message from above claims, that no Authorization header has been sent. Did i miss something here?

Furthermore, i didn’t really understood the dependencies between config.yml and opensearch-dashboards.yml and why i need to configure e.g. the connect url two times.

Any help or hint is highly appreciated

Configuration:

opensearch-dashboards.yml

server.host: "0.0.0.0"
server.name: opensearch-dashboards
opensearch.username: "admin"
opensearch.password: "admin"
server.ssl.enabled: true
server.ssl.certificate: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.pem"
server.ssl.key: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key"
opensearch.ssl.certificateAuthorities: ["/usr/share/opensearch-dashboards/config/certificates/ca/ca.pem"]
opensearch.ssl.verificationMode: none
opensearch_security.cookie.secure: true
opensearch.requestHeadersAllowlist: ["Authorization"]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Global"]
 #enabled both to see if it is still working
opensearch_security.auth.type: ["basicauth","openid"]
opensearch_security.auth.multiple_auth_enabled: true
opensearch_security.openid.connect_url: "https://thefancyprovider/.well-known/openid-configuration"
opensearch_security.openid.client_id: "<censored>"
opensearch_security.openid.client_secret: "<censored>"
opensearch_security.openid.scope: "openid entitlement_group"
opensearch_security.openid.header: "Authorization"
opensearch_security.openid.logout_url: "https://localhostlogouturl/logout"
opensearch_security.cookie.isSameSite: None
opensearch_security.openid.verify_hostnames: false
opensearch_security.openid.base_redirect_url: "https://localhost:5601"
opensearch_security.openid.trust_dynamic_headers: true
opensearch_security.openid.refresh_tokens: true

config.yml

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        description: "OpenID connection"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: sub
            roles_key: entitlement_group
            openid_connect_url: https://thefancyprovider/.well-known/openid-configuration
            skip_users:
              - admin
        authentication_backend:
          type: noop

Relevant Logs or Screenshots:

B3n · April 25, 2023, 6:53am

Okay well, it was as i thought. I had to map the claim sub to my internal_users.yml file.

inside the sub it looked like the following:

sub: USERID

But i didn’t thought about any case sensitivity. As my internal_users.yml config looked like:

userid:
hash: “”
[…]

so i’ve changed from userid to USERID and now i can login with my user.

Update:

That way i can securely deactivate the hash, right? As a password is not necessary anymore?
Even if the login is working, i’m receiving the following errors:

opensearch-node1         | [2023-04-25T07:00:19,339][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-node1         | [2023-04-25T07:00:19,340][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node1] Failed to get roles from JWT claims with roles_key 'entitlement_group'. Check if this key is correct and available in the JWT payload.
opensearch-node3         | [2023-04-25T07:00:19,343][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node3] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-node3         | [2023-04-25T07:00:19,344][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node3] Failed to get roles from JWT claims with roles_key 'entitlement_group'. Check if this key is correct and available in the JWT payload.
opensearch-node2         | [2023-04-25T07:00:19,348][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node2] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-node2         | [2023-04-25T07:00:19,348][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node2] Failed to get roles from JWT claims with roles_key 'entitlement_group'. Check if this key is correct and available in the JWT payload.

pablo · April 25, 2023, 9:03am

@B3n You don’t need internal users with OIDC authentication. The idea is to have your OIDC users managed by an external IdP and not the OpenSearch cluster.

The missing part is the lack of role mapping. The groups/roles assigned to the user in the IdP should be mapped to the OpenSearch’s roles as backend roles.
To achieve that you need to configure that mapping in the roles_mapping.yml file.

What is your IdP?

B3n · April 25, 2023, 9:29am

Thanks for your reply @pablo

I’m using Ping Identity as IdP.

For my own understanding: When i want to use an external OIDC provider i have to define roles with their specific permissions and then define the mappings inside roles_mapping.yml with backend_roles that match to the roles coming from the IdP?

As i’m using a docker-compose setup where i include the file “internal_users.yml”, this makes the include of this file obsolete, right?

So i suppose the challenge here is to get the roles correctly out of the IdP. Usually the roles are entitlements and are stored inside the claim “entitlement_group”

But the error e.g.

opensearch-node2 | [2023-04-25T07:00:19,348][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node2] Failed to get roles from JWT claims with roles_key 'entitlement_group'. Check if this key is correct and available in the JWT payload.

confuses me. Can i somehow enable a logging to see the actual payload coming from the IdP?

pablo · April 25, 2023, 1:51pm

Correct.

Correct, unless you wish to have some custom internal users (i.e. logstash, filebeat, metricbeat etc.)

Correct.

This error states that entitlement_group was not found in the JWT token sent by the IdP. Therefore, the security plugin couldn’t map the IdP roles with the OpenSearch roles.

Unfortunately, OpenSearch is lacking OpenID token logging.
Have you tried the approach described in this PingIdentity document?

https://docs.pingidentity.com/r/en-us/pingfederate-egnyte-connector/pingfederate_egnyte_connector_obtain_access_token

B3n · April 26, 2023, 6:12am

Thanks a lot for your clarification!

Unfortunately i do not have any access to the ping id setup, and i only have limited access in general. So i suppose i can’t dig any deeper on that and use anything roles_key related

pablo · April 26, 2023, 8:27am

@B3n There is another option. You can use PingID username instead of the role name in the roles_mapping.yml

i.e.

custom_role:
  reserved: false
  users:
  - "<pingid_user>"

It’s not a perfect solution but at least will keep you away from internal users. This user field accepts also regular expressions. For example, if your users have a common phrase in their usernames then you can make this role assignment more flexible.

i.e.

custom_role:
  reserved: false
  users:
  - "*"

B3n · April 26, 2023, 10:50am

Hey @pablo!

That works perfectly fine for me, thanks a lot for your input and help!

Topic		Replies	Views
"statusCode":401 while redirecting back to opensearch-dashboard using OIDC Security troubleshoot , configure	5	518	March 26, 2024
OpenSearch Dashboards 401 Unauthorized using OIDC with Keycloak Security	12	6373	February 15, 2023
Unauthorized using azure oidc jwt OpenSearch security-issue	5	354	June 22, 2023
Authorization issue using OpenID (Keycloak) Security	17	1295	November 13, 2023
Issue with OpenID Connect (OIDC) and TLS Security security-issue	6	848	April 4, 2023

OIDC Integration: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

Related topics