Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch: 2.11.0 (I use the tag latest)
Opensearch-Dashboard: 2.11.0 (I use the tag latest)
Windows 10
Docker Desktop 4.25.0
Tested with Edge 119.0 and Chrome 119.0
Keycloak 22.0.5
Describe the issue:
I’m trying to integrate opensearch and opensearch-dashboards, launched on my laptop using docker compose, and a keycloak instance located in another machine.
The keycloak is available using the following address (http://keycloak:8080). I know this is http but this is only a test setup.
When I try to connect, I am able to enter my credentials but then, opensearch-dashboards return me the following error:
{
"statusCode": 401,
"error": "Unauthorized",
"message": "Unauthorized"
}
I changed the log level to verbose and I activated the security traces as proposed in Troubleshoot OpenID Connect - OpenSearch documentation.
The opensearch-dashboard logs are:
2023-11-13 11:34:03 {"type":"log","@timestamp":"2023-11-13T10:34:03Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}
2023-11-13 11:34:03 {"type":"response","@timestamp":"2023-11-13T10:34:03Z","tags":[],"pid":1,"method":"get","statusCode":401,"req":{"url":"/auth/openid/login?state=VRHx2nGfYq4P3dgS9HdBB_&session_state=84c63e91-86e8-4095-87a3-ca798ebd5649&code=63fa4acb-2458-4c3b-a2d1-6b40cdd277d2.84c63e91-86e8-4095-87a3-ca798ebd5649.c0d53dac-7633-4c24-b2ac-9f05e97df2ff","method":"get","headers":{"host":"localhost:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"none","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","sec-ch-ua":"\"Microsoft Edge\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","accept-encoding":"gzip, deflate, br","accept-language":"en,fr;q=0.9,fr-FR;q=0.8,en-GB;q=0.7,en-US;q=0.6"},"remoteAddress":"172.23.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0"},"res":{"statusCode":401,"responseTime":100,"contentLength":9},"message":"GET /auth/openid/login?state=VRHx2nGfYq4P3dgS9HdBB_&session_state=84c63e91-86e8-4095-87a3-ca798ebd5649&code=63fa4acb-2458-4c3b-a2d1-6b40cdd277d2.84c63e91-86e8-4095-87a3-ca798ebd5649.c0d53dac-7633-4c24-b2ac-9f05e97df2ff 401 100ms - 9.0B"}
and the opensearch traces (with trace level) are:
2023-11-13 11:34:03 [2023-11-13T10:34:03,090][TRACE][o.o.i.b.request ] [opensearch-node1] [request] Adjusted breaker by [-16440] bytes, now [0]
2023-11-13 11:34:03 [2023-11-13T10:34:03,420][TRACE][o.o.i.IndexService ] [opensearch-node1] [.kibana_1] scheduling refresh every 1s
2023-11-13 11:34:03 [2023-11-13T10:34:03,426][TRACE][o.o.i.IndexService ] [opensearch-node1] [.opensearch-sap-log-types-config] scheduling refresh every 1s
2023-11-13 11:34:03 [2023-11-13T10:34:03,429][TRACE][o.o.i.IndexService ] [opensearch-node1] [.opensearch-observability] scheduling refresh every 1s
2023-11-13 11:34:03 [2023-11-13T10:34:03,430][TRACE][o.o.h.AbstractHttpServerTransport] [opensearch-node1] Http channel accepted: Netty4HttpChannel{localAddress=/172.23.0.5:9200, remoteAddress=/172.23.0.2:38364}
2023-11-13 11:34:03 [2023-11-13T10:34:03,432][TRACE][o.o.i.IndexService ] [opensearch-node1] [security-auditlog-2023.11.10] scheduling refresh every 1s
2023-11-13 11:34:03 [2023-11-13T10:34:03,437][DEBUG][i.n.h.s.SslHandler ] [opensearch-node1] [id: 0x8bd7336f, L:/172.23.0.5:9200 - R:/172.23.0.2:38364] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_256_GCM_SHA384
2023-11-13 11:34:03 [2023-11-13T10:34:03,437][TRACE][o.o.s.h.XFFResolver ] [opensearch-node1] resolve /172.23.0.2:38364
2023-11-13 11:34:03 [2023-11-13T10:34:03,437][TRACE][o.o.s.h.XFFResolver ] [opensearch-node1] no xff done (enabled or no netty request) false,class org.opensearch.security.filter.NettyRequestChannel,{},{}
2023-11-13 11:34:03 [2023-11-13T10:34:03,437][TRACE][o.o.s.a.BackendRegistry ] [opensearch-node1] Rest authentication request from 172.23.0.2:38364 [original: /172.23.0.2:38364]
2023-11-13 11:34:03 [2023-11-13T10:34:03,438][DEBUG][o.o.s.a.BackendRegistry ] [opensearch-node1] Check authdomain for rest internal/4 or 1 in total
2023-11-13 11:34:03 [2023-11-13T10:34:03,438][TRACE][o.o.s.a.BackendRegistry ] [opensearch-node1] Try to extract auth creds from basic http authenticator
2023-11-13 11:34:03 [2023-11-13T10:34:03,438][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
2023-11-13 11:34:03 [2023-11-13T10:34:03,438][TRACE][o.o.s.a.i.AuditLogImpl ] [opensearch-node1] Check for REST category:FAILED_LOGIN, effectiveUser:<NONE>, request:/_plugins/_security/authinfo
2023-11-13 11:34:03 [2023-11-13T10:34:03,439][TRACE][o.o.s.a.r.AuditMessageRouter] [opensearch-node1] will store on sink InternalOpenSearchSink asynchronously
2023-11-13 11:34:03 [2023-11-13T10:34:03,439][TRACE][o.o.s.a.BackendRegistry ] [opensearch-node1] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'
2023-11-13 11:34:03 [2023-11-13T10:34:03,440][TRACE][o.o.h.HttpTracer ] [opensearch-node1] [15][d4a8de2f-cebf-449f-9dae-2b417d32a168][GET][/_plugins/_security/authinfo] received request from [Netty4HttpChannel{localAddress=/172.23.0.5:9200, remoteAddress=/172.23.0.2:38364}]
2023-11-13 11:34:03 [2023-11-13T10:34:03,440][TRACE][o.o.i.b.in_flight_requests] [opensearch-node1] [in_flight_requests] Adding [0b][<http_request>] to used bytes [new used: [0b], limit: 536870912 [512mb], estimate: 0 [0b]]
2023-11-13 11:34:03 [2023-11-13T10:34:03,440][TRACE][o.o.i.b.in_flight_requests] [opensearch-node1] [in_flight_requests] Adjusted breaker by [0] bytes, now [0]
2023-11-13 11:34:03 [2023-11-13T10:34:03,441][TRACE][o.o.h.HttpTracer ] [opensearch-node1] [15][d4a8de2f-cebf-449f-9dae-2b417d32a168][UNAUTHORIZED][text/plain; charset=UTF-8][0] sent response to [Netty4HttpChannel{localAddress=/172.23.0.5:9200, remoteAddress=/172.23.0.2:38364}] success [true]
2023-11-13 11:34:03 [2023-11-13T10:34:03,441][TRACE][o.o.t.TaskManager ] [opensearch-node1] register 157 [transport] [indices:data/write/index] [index {[security-auditlog-2023.11.13][null], source[{"audit_cluster_name":"opensearch-cluster","audit_node_name":"opensearch-node1","audit_rest_request_method":"GET","audit_category":"FAILED_LOGIN","audit_request_origin":"REST","audit_node_id":"Kt3r0fC2SSmTKIILoU2wXg","audit_request_layer":"REST","audit_rest_request_path":"/_plugins/_security/authinfo","@timestamp":"2023-11-13T10:34:03.438+00:00","audit_request_effective_user_is_admin":false,"audit_format_version":4,"audit_request_remote_address":"172.23.0.2","audit_node_host_address":"172.23.0.5","audit_rest_request_headers":{"x-opensearch-product-origin":["opensearch-dashboards"],"Connection":["keep-alive"],"x-opaque-id":["d4a8de2f-cebf-449f-9dae-2b417d32a168"],"Host":["opensearch:9200"],"Content-Length":["0"]},"audit_request_effective_user":"<NONE>","audit_node_host_name":"172.23.0.5"}]}]
2023-11-13 11:34:03 [2023-11-13T10:34:03,441][TRACE][opendistro_security_action_trace] [opensearch-node1] Node opensearch-node1 -> indices:data/write/index (): userIsAdmin=false/conRequest=true/internalRequest=falseorigin=LOCAL/directRequest=true/remoteAddress=null
2023-11-13 11:34:03 [2023-11-13T10:34:03,442][TRACE][opendistro_security_action_trace] [opensearch-node1] Node opensearch-node1 -> indices:data/write/bulk (1): userIsAdmin=false/conRequest=true/internalRequest=falseorigin=LOCAL/directRequest=true/remoteAddress=null
2023-11-13 11:34:03 [2023-11-13T10:34:03,450][TRACE][o.o.t.TaskManager ] [opensearch-node1] register 158 [transport] [indices:data/write/bulk[s]] [requests[1], index[security-auditlog-2023.11.13][0], refresh[IMMEDIATE]]
2023-11-13 11:34:03 [2023-11-13T10:34:03,451][TRACE][opendistro_security_action_trace] [opensearch-node1] Node opensearch-node1 -> indices:data/write/bulk[s] (): userIsAdmin=false/conRequest=true/internalRequest=falseorigin=LOCAL/directRequest=true/remoteAddress=null
The user I’m trying to connect with has the “all_access” role.
I tryed to followed all the tuto and troubleshooting I found but I have no idea where my error can be.
Thanks in advance for your help!
Configuration:
- Opensearch security configuration:
_meta:
type: "config"
config_version: 2
config:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: false
internalProxies: '192\.168\.0\.10|192\.168\.0\.11'
basic_internal_auth_domain:
description: "Authenticate via HTTP Basic against internal users database"
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
openid_auth_domain:
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: openid
challenge: false
config:
subject_key: preferred_username
roles_key: roles
openid_connect_url: http://keycloak:8080/realms/Remi_Support_Prototype/.well-known/openid-configuration
jwt_header: Authorization
openid_connect_idp:
enable_ssl: false
authentication_backend:
type: noop
- Opensearch dashboard configuration:
opensearch.hosts: [https://localhost:9200]
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: [authorization, securitytenant]
opensearch_security.multitenancy.enabled: false
opensearch_security.readonly_mode.roles: [kibana_read_only]
# Use this setting if you are running opensearch-dashboards without https
opensearch_security.cookie.secure: false
server.host: '0.0.0.0'
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: http://keycloak:8080/realms/Remi_Support_Prototype/.well-known/openid-configuration
opensearch_security.openid.client_id: opensearch
opensearch_security.openid.client_secret: QumkJeSWXBJJu3AczsDlB2Vt68CTz6lw
opensearch_security.openid.verify_hostnames: false
opensearch_security.openid.refresh_tokens: false
opensearch_security.openid.base_redirect_url: "http://localhost:5601"
logging.verbose: true
- Keycloak client configuration:
{
"clientId": "opensearch",
"name": "",
"description": "",
"rootUrl": "http://localhost:5601/",
"adminUrl": "",
"baseUrl": "",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "QumkJeSWXBJJu3AczsDlB2Vt68CTz6lw",
"redirectUris": [
"http://localhost:5601/*"
],
"webOrigins": [
"http://localhost:5601"
],
"notBefore": 0,
"bearerOnly": false,
"consentRequired": false,
"standardFlowEnabled": true,
"implicitFlowEnabled": true,
"directAccessGrantsEnabled": true,
"serviceAccountsEnabled": true,
"authorizationServicesEnabled": true,
"publicClient": false,
"frontchannelLogout": true,
"protocol": "openid-connect",
"attributes": {
"oidc.ciba.grant.enabled": "false",
"client.secret.creation.time": "1699864768",
"backchannel.logout.session.required": "true",
"oauth2.device.authorization.grant.enabled": "false",
"display.on.consent.screen": "false",
"backchannel.logout.revoke.offline.tokens": "false"
},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": true,
"nodeReRegistrationTimeout": -1,
"protocolMappers": [
{
"name": "client roles",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-client-role-mapper",
"consentRequired": false,
"config": {
"multivalued": "true",
"userinfo.token.claim": "true",
"user.attribute": "foo",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "roles",
"jsonType.label": "String"
}
},
{
"name": "Client IP Address",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "clientAddress",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "clientAddress",
"jsonType.label": "String"
}
},
{
"name": "Client ID",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "client_id",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "client_id",
"jsonType.label": "String"
}
},
{
"name": "Client Host",
"protocol": "openid-connect",
"protocolMapper": "oidc-usersessionmodel-note-mapper",
"consentRequired": false,
"config": {
"user.session.note": "clientHost",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "clientHost",
"jsonType.label": "String"
}
}
],
"defaultClientScopes": [
"web-origins",
"acr",
"roles",
"profile",
"email"
],
"optionalClientScopes": [
"address",
"phone",
"offline_access",
"microprofile-jwt"
],
"access": {
"view": true,
"configure": true,
"manage": true
}
}
Relevant Logs or Screenshots:
