Unable to send notification with configured OIDC

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.8.0

Describe the issue: When OIDC auth is enabled test/normal notification can’t be sent. Login itself, views, and other permissions work as expected. There are no issues with the group/role assigned by the idp.

Configuration:

Dashboards

      opensearch.ssl.verificationMode: "Full"
      opensearch_security.auth.anonymous_auth_enabled: "false"
      opensearch_security.auth.type: |
        ["openid", "basicauth"]
      opensearch.requestHeadersAllowlist: |
        ["authorization", "securitytenant"]
      opensearch_security.multitenancy.tenants.preferred: |
        ["global", "customer"]
      opensearch_security.multitenancy.tenants.enable_global: "true"
      opensearch_security.multitenancy.tenants.enable_private: "false"
      opensearch_security.multitenancy.enable_aggregation_view: "false"
      opensearch_security.openid.trust_dynamic_headers: "true"
      opensearch_security.multitenancy.enabled: "true"
      opensearch_security.openid.connect_url: "${OPENID_CONNECT_URL}"
      opensearch_security.openid.client_id: "${OPENID_CLIENT_ID}"
      opensearch_security.openid.client_secret: "${OPENID_CLIENT_SECRET}"
      opensearch_security.openid.scope: "openid profile name groups"
      opensearch_security.auth.multiple_auth_enabled: "true"

OS

      node.search.cache.size: 500m
      cluster.max_shards_per_node: "10000"
      prometheus.indices: "false"
      reindex.remote.allowlist: "elasticsearch.elastic-stack-logging:9200, elasticsearch:9200, localhost:9200"
      plugins.alerting.filter_by_backend_roles: "true"
      opensearch.notifications.general.filter_by_backend_roles: "true"

Relevant Logs or Screenshots:

$ curl -XGET "https://opensearch.elastic-stack-logging.svc.cluster.local:9200/_plugins/_security/authinfo?pretty"
{
  "user": "User [name=username, backend_roles=[list, of, the, roles], requestedTenant=]",
  "user_name": "username",
  "user_requested_tenant": "",
  "remote_address": "10.1.5.199:59582",
  "backend_roles": [
    "list",
    "of",
    "the",
    "roles"
  ],
  "custom_attribute_names": [
    "attr.jwt.iss",
    "attr.jwt.auth_time",
    "attr.jwt.org",
    "attr.jwt.amr",
    "attr.jwt.aud",
    "attr.jwt.acr",
    "attr.jwt.groups",
    "attr.jwt.at_hash",
    "attr.jwt.env",
    "attr.jwt.sid",
    "attr.jwt.sub",
    "attr.jwt.name",
    "attr.jwt.exp",
    "attr.jwt.iat",
    "attr.jwt.p1.region"
  ],
  "roles": [
    "all_access"
  ],
  "tenants": {
    "global_tenant": true,
    "username": true,
    "customer": true
  },
  "principal": null,
  "peer_certificates": "0",
  "sso_logout_url": null
}
[2024-04-02T14:34:22,621][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-04-02T14:34:22,623][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-04-02T14:34:22,839][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-04-02T14:34:22,843][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-04-02T14:34:22,845][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-04-02T14:34:22,846][INFO ][o.o.n.r.NotificationConfigRestHandler] [opensearch-cluster-hot-0] notifications:executeGetRequest from:0, maxItems:1000, sortField:null, sortOrder=null, filters={}
[2024-04-02T14:34:22,846][INFO ][o.o.n.i.ConfigIndexingActions] [opensearch-cluster-hot-0] notifications:NotificationConfig-get org.opensearch.commons.notifications.action.GetNotificationConfigRequest@6af51d
[2024-04-02T14:34:22,847][INFO ][o.o.n.i.ConfigIndexingActions] [opensearch-cluster-hot-0] notifications:NotificationConfig-info HLkqn44B1gJovatpNwFO
[2024-04-02T14:34:22,850][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-cluster-hot-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'

@kaliha What type of channel did you use?
What’s the name of OpenID IdP provider?

Did you upgrade your environment or switch from basicauth to OIDC?

1 Like

SNS

Private one

Clean cluster, started with 2 auth types basic and oidc with order 0 and 1

Addition: We use IRSA, and opensearch doesn’t pick up the IRSA role and tries to use node role.

@kaliha How did you deploy yiur cluster? Is it AWS managed OpenSearch?
Where did you try to use that IAM role?

I deploy using the os operator. IAM role is used to allow sent notifications from the OS to SNS. The issue in IRSA role usage in the opensearch-notification plugin.

So it’s not related to the OIDC, simply wrong error codes/messages when plugin fails.