Missing `security_authentication` cookie kibana page will fail with 401

Security
1

Hello,
I have deployed on opensearch cluster with oidc auth and we get {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"} error from time to time.

From what I found, if security_authentication cookie is not present, opensearch doesn’t use OIDC auth and maybe use internal_auth which will fail because no internal user is used.

The error returned is {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"} .
Maybe is related with this part security-dashboards-plugin/security_cookie.ts at e08dca4eca8906ade344a00ca2b80289037522c3 · opensearch-project/security-dashboards-plugin · GitHub.

Created github issue here [BUG] Missing `security_authentication` cookie auth fails with 401 · Issue #939 · opensearch-project/security-dashboards-plugin · GitHub

Thank you.

2

@H1gH Could you share your opensearch_dashboards.yml file?

3

Sure @pablo,

server.defaultRoute: "/app/dashboards/?security_tenant=global"
server.name: opensearch-dashboards
server.host: "0"
server.ssl.enabled: false
server.ssl.certificate: /usr/share/opensearch_dashboards/config/ssl/node.pem
server.ssl.key: /usr/share/opensearch_dashboards/config/ssl/node.key
opensearch.hosts: https://cluster-url:9200
opensearch.ssl.certificateAuthorities: [ "/usr/share/opensearch_dashboards/config/opensearch-certs/ca.pem" ]
opensearch.ssl.verificationMode: certificate
opensearch.username: kibanaserver
opensearch.password: SuP3rS3cuR3P@ssW0rd
opensearch.requestHeadersWhitelist: ["securitytenant", "Authorization"]

opensearch_security.multitenancy.enabled: false
opensearch_security.readonly_mode.roles: ["Dashboard User"]

opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://oauth-url/.well-known/openid-configuration"
opensearch_security.openid.client_id: "client_id-sfsdfdf"
opensearch_security.openid.client_secret: "client_secretsss"
opensearch_security.openid.base_redirect_url: "https://opensearch-dashboards-url.net"
opensearch_security.openid.verify_hostnames: false
opensearch_security.openid.scope: offline_access
opensearch_security.session.keepalive: false

opensearchDashboards.branding:
  applicationTitle: "Opensearch Dashboards"
  faviconUrl: "https://opensearch-dashboards-url.net/ui/favicon.png"
  logo:
    defaultUrl: "https://opensearch-dashboards-url.net/ui/logo.png"
  mark:
    defaultUrl: "https://opensearch-dashboards-url.net/ui/favicon.png"
  loadingLogo:
    defaultUrl: "https://opensearch-dashboards-url.net/ui/favicon.png"
4

@H1gH Do you use HA Proxy to connect to OpenSearch Dashboards? If so, do you use HTTPS or HTTP between proxy and OpenSearch Dashboards?

5

@pablo Yes, we are using a http proxy between exposed url and opensearch dashboards.

6

@H1gH Do you balance OpenSearch Dashboards? I mean do you have more than one OpenSearch Dashboards instance behind the proxy?

7

@pablo No, it’s only in one instance deployed

8

@H1gH Would you mind sharing config.yml?

9

@pablo, sure

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: true
          config:
            subject_key: preferred_username
            roles_key: teams
            openid_connect_idp:
              enable_ssl: false
              verify_hostnames: false
            openid_connect_url: https://oauth-url.net/.well-known/openid-configuration
        authentication_backend:
          type: noop
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
10

@H1gH You’ve set enable_ssl: false but you still trying to connect with HTTPS

openid_connect_url: https://oauth-url.net/.well-known/openid-configuration

11

@pablo The thing is that is working as expected, it’s working with enable_ssl: false
Only problem is that sometime we get 401 error and we found out is related with missing cookie.
Could this be related with our problem?

12

@H1gH opensearch.hosts: https://cluster-url:9200 Do you point the proxy here?

13

@pablo No, is kubernetes expose

14

I think it’s design flaw of Opensearch Dashboards’ client side session. Below are my guess,

  • It stores access token, refresh token and id token in the cookie
  • It refresh all tokens once the id token is about to expire, which could have very short life span, e.g. 2 minutes.
  • If there are simultaneous http request from the UI when the id token expires, the same cookie session are sent to backend and try to refresh tokens twice with the same refresh token, and the latter refresh request will be invalid as the its refresh token is invalidated because it’s been used by the first request.
    I don’t think there is an easy solution for client-side session. Maybe there can be a tolerant-for-refresh-failure setting that can ignore the randomly happened issue.