Configuring OIDC authentication with EKS OIDC provider

Detected OpenSearch Version: x-content-2.9.0
Detected OpenSearch Security Version:

I am trying to configure OIDC authentication by using the Open ID Connect endpoint provided by EKS. I hope to use the JWT token that is created for a Kubernetes service account to authenticate to OpenSearch. This means I would not need to maintain passwords in our services.

I have created an entry for the user that matches the “sub” of the JWT token.


      config.yml: |- 
          type: "config"
          config_version: 2

              anonymous_auth_enabled: false
                enabled: false
                internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
                description: "Authenticate via HTTP Basic against internal users database"
                http_enabled: true
                transport_enabled: true
                order: 0
                  type: basic
                  challenge: true
                  type: intern
                http_enabled: true
                transport_enabled: true
                order: 1
                  type: openid
                  challenge: false
                    subject_key: preferred_username
                    roles_key: roles
                    verify_hostnames: false
                  type: noop
      internal_users.yml: |-
          type: "internalusers"
          config_version: 2
          hash: "$2y$12$eRDrPGUTb95FgBt/PlN5QODnEsbTxU2f2cVp5jEFxuKomLFohKsW6"
          reserved: true
          - "admin"
          description: "Admin user"
          hash: "$2y$12$eRDrPGUTb95FgBt/PlN5QODnEsbTxU2f2cVp5jEFxuKomLFohKsW6"
          reserved: false
          - "admin"
          description: "open-search-poc-updater service account"

The decoded JWT token looks like this:

  "aud": [
  "exp": 1724264026,
  "iat": 1692728026,
  "iss": "",
  "": {
    "namespace": "default",
    "pod": {
      "name": "test",
      "uid": "XYZZY"
    "serviceaccount": {
      "name": "open-search-poc-updater",
      "uid": "XYZZY"
    "warnafter": 1692731633
  "nbf": 1692728026,
  "sub": "system:serviceaccount:default:open-search-poc-updater"

I am able to get the JWT token and include it as a Bearer token, but I get an Unauthorized result.

Should this work? Am I missing something in configuration

Relevant Logs or Screenshots:

curl --no-progress-meter -v -k -H "Authorization: Bearer $token" "https://open-search-poc.default.svc.cluster.local:9200/?pretty" 
* processing: https://open-search-poc.default.svc.cluster.local:9200/?pretty
*   Trying
* Connected to open-search-poc.default.svc.cluster.local ( port 9200
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: DC=de; L=test; O=node; OU=node;
*  start date: Apr 22 03:43:47 2018 GMT
*  expire date: Apr 19 03:43:47 2028 GMT
*  issuer: DC=com; DC=example; O=Example Com Inc.; OU=Example Com Inc. Root CA; CN=Example Com Inc. Root CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* using HTTP/1.x
> GET /?pretty HTTP/1.1
> Host: open-search-poc.default.svc.cluster.local:9200
> User-Agent: curl/8.2.1
> Accept: */*
> Authorization: Bearer eyJhb<redacted>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm="OpenSearch Security"
< content-type: text/plain; charset=UTF-8
< content-length: 12
* Connection #0 to host open-search-poc.default.svc.cluster.local left intact

I see this in the OpenSearch logs:

[2023-08-22T19:09:32,662][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-2] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,736][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-1] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,740][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-1] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,741][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-1] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,749][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-2] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,785][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-0] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,824][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-0] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:32,825][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-0] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:36,789][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-master-1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2023-08-22T19:09:36,791][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-1] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'

I think the “No ‘Authorization’ header” warnings are from communications between nodes in the cluster. The “No ‘Basic Authorization’ header” seems to be from my attempt to curl to the cluster.