OpenID Connect Authentication Issue

Security
1

I’m attempting to use OIDC (Keycloak IdP) for user SSO in Kibana. My security config is as follows:

    basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://domain/auth/realms/myrealm/.well-known/openid-configuration
            enable_ssl: true
            verify_hostnames: false
        authentication_backend:
          type: noop

And my kibana config:

elasticsearch.hosts: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization", "security_tenant"]

opendistro_security.auth.type: "openid"
opendistro_security.openid.connect_url: "https://domain/auth/realms/myrealm/.well-known/openid-configuration"
opendistro_security.openid.client_id: "elasticsearch"
opendistro_security.openid.client_secret: "secret"
opendistro_security.openid.base_redirect_url: "https://my.tld"

opendistro_security.cookie.secure: true
opendistro_security.cookie.password: "redacted"

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]

newsfeed.enabled: false
telemetry.optIn: false
telemetry.enabled: false

However, upon loading the kibana dashboard, I get the error:
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}.

The corresponding Kibana log entry:

Jul 23 19:10:11 ubuntu1804 kibana[1445]: {"type":"log","@timestamp":"2020-07-23T19:10:11Z","tags":["error","http"],"pid":1445,"message":"{ Error: Authentication Exception\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)\n    at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)\n    at IncomingMessage.emit (events.js:203:15)\n    at endReadableNT (_stream_readable.js:1145:12)\n    at process._tickCallback (internal/process/next_tick.js:63:19)\n  status: 401,\n  displayName: 'AuthenticationException',\n  message: 'Authentication Exception',\n  path: '/.kibana/_doc/config%3A7.8.0',\n  query: {},\n  body: 'Authentication finally failed',\n  statusCode: 401,\n  response: 'Authentication finally failed',\n  toString: [Function],\n  toJSON: [Function],\n  isBoom: true,\n  isServer: false,\n  data: null,\n  output:\n   { statusCode: 401,\n     payload:\n      { statusCode: 401,\n        error: 'Unauthorized',\n        message: 'Authentication Exception' },\n     headers:\n      { 'WWW-Authenticate': 'Basic realm=\"Authorization Required\"' } },\n  reformat: [Function],\n  [Symbol(ElasticsearchError)]: 'Elasticsearch/notAuthorized',\n  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/notAuthorized' }"}
Jul 23 19:10:11 ubuntu1804 kibana[1445]: {"type":"error","@timestamp":"2020-07-23T19:10:11Z","tags":[],"pid":1445,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toInternalError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:69:19)\n    at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:163:34)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/","path":"/","href":"/"},"message":"Internal Server Error"}

And finally, the log for elastic:

[2020-07-23T20:00:37,915][TRACE][c.a.o.s.h.XFFResolver    ] [ubuntu1804] no xff done (enabled or no netty request) false,class org.elasticsearch.rest.RestRequest,{},{}
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Rest authentication request from 127.0.0.1:57104 [original: /127.0.0.1:57104]
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] Check authdomain for rest internal/0 or 2 in total
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Try to extract auth creds from basic http authenticator
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] No 'Authorization' header, send 403
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] Check authdomain for rest noop/1 or 2 in total
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Try to extract auth creds from jwt-key-by-oidc http authenticator
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] No 'Authorization' header, send 403
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] User still not authenticated after checking 2 auth domains
[2020-07-23T20:00:37,915][WARN ][c.a.o.s.a.BackendRegistry] [ubuntu1804] Authentication finally failed for null from 127.0.0.1:57104
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.i.AuditLogImpl ] [ubuntu1804] Check for REST category:FAILED_LOGIN, effectiveUser:null, request:/.kibana/_doc/config:7.8.0

Any help would be appreciated - thank you!

2

Have you tried this OpenID Troubleshooting | Security for Elasticsearch | Search Guard

3

I’m not using Search Guard, but I did try all of Open Distro’s OIDC debugging steps.

4

I have the exact same problem.

In my case the only diference is i dont have any elasticsearch logs.
Are you using the Helm chart? and how did you turned the logs on?

5

I’m not sure what a Helm chart is so I guess not. Logging configuration is in /etc/elasticsearch/log4j2.properties. I ended up not using Open Distro and using GitHub - novomatic-tech/keycloak-kibana: A Keycloak authorization plugin for Kibana instead.

6

Anyone using helm charts that can provide the OIDC configuration that is being passed on the values.yaml file ?
Much appreciated

7

I am using the same configurations that were posted on this post, but they dont work, even if they are the ones on the documentation

8

I resolved my problem, It was being cause by using persistance in the cloud. For some reason elk was selectively choosing some settings and removing others.

Without persistance it runs all configurations i send him and was able to connect to ldap.

9

Looks were similar to issue I still have:

I just didn’t get that much info in logs.

10

I’m struggling with the same issue (elasticsearch-oss-7.8.0 opendistro_security-1.9.0.2 kibana-oss-7.8.0 opendistro_security_kibana_plugin-1.9.0.0)

I figured out that this issue only appears when kibana uses https for connection to OIDC provider (keycloak in my case) eg opendistro_security.openid.connect_url

11

Figured out underlying issue - kibana security plugin fails to initialize due to un-trusted OIDC provider’s certificate - fix is simple export NODE_TLS_REJECT_UNAUTHORIZED=0

12

After a bit more digging found out that proper certificate validation is already implemented and hides under undocumented parameter “opendistro_security.openid.root_ca” just point it to trust chain certificate in pem format and it should work.

On the other side kibana plugins initialization swallows all exceptions during authentication backed initialization, for this I’m using

— index.js.without.details 2020-09-14 03:33:19.092713615 +0000
+++ index.js 2020-09-14 03:36:37.015868631 +0000
@@ -399,6 +399,7 @@
// At the moment this is mainly to catch an error where the openid connect_url is > wrong
await authClass.init();
} catch (error) {
+ this.status.red(‘Authentication backend initialization failed with:’ + error.message);
this.status.red(‘An error occurred during initialisation, please check the logs.’);
return;
}

https://github.com/opendistro-for-elasticsearch/security-kibana-plugin/pull/28