Good call! I missed that in the documentation.
I also noticed that I had an indentation error in the YAML config above.
After fixing both and running running securityadmin.sh:
[opensearch@15fdeb566e3a tools]$ ./securityadmin.sh -cd ../../../config/opensearch-security/ -icl -nhnv \
> -cacert ../../../config/root-ca.pem \
> -cert ../../../config/kirk.pem \
> -key ../../../config/kirk-key.pem
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.2.1
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch-cluster
Clusterstate: GREEN
Number of nodes: 2
Number of data nodes: 2
.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!
Populate config from /usr/share/opensearch/config/opensearch-security
Will update '/config' with ../../../config/opensearch-security/config.yml (legacy mode)
SUCC: Configuration for 'config' created or updated
Will update '/roles' with ../../../config/opensearch-security/roles.yml (legacy mode)
SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with ../../../config/opensearch-security/roles_mapping.yml (legacy mode)
SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with ../../../config/opensearch-security/internal_users.yml (legacy mode)
SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with ../../../config/opensearch-security/action_groups.yml (legacy mode)
SUCC: Configuration for 'actiongroups' created or updated
Will update '/nodesdn' with ../../../config/opensearch-security/nodes_dn.yml (legacy mode)
SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with ../../../config/opensearch-security/whitelist.yml (legacy mode)
SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with ../../../config/opensearch-security/audit.yml (legacy mode)
SUCC: Configuration for 'audit' created or updated
SUCC: Expected 7 config types for node {"updated_config_types":["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"],"updated_config_size":7,"message":null} is 7 (["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"]) due to: null
SUCC: Expected 7 config types for node {"updated_config_types":["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"],"updated_config_size":7,"message":null} is 7 (["config","roles","rolesmapping","internalusers","actiongroups","nodesdn","audit"]) due to: null
Done with success
So far so good!
Now, trying to access OSD we get a different error (progress, yay!):
opensearch-node2 | [2022-09-19T14:25:17,985][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node2] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-node2 | [2022-09-19T14:25:18,041][INFO ][c.a.d.a.h.j.k.SelfRefreshingKeySet] [opensearch-node2] Performing refresh 1
opensearch-node2 | [2022-09-19T14:25:18,166][INFO ][c.a.d.a.h.j.k.SelfRefreshingKeySet] [opensearch-node2] KeySetProvider finished
opensearch-node2 | [2022-09-19T14:25:18,168][INFO ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node2] Extracting JWT token from XXXXX failed
opensearch-node2 | com.amazon.dlic.auth.http.jwt.keybyoidc.BadCredentialsException: Unknown kid 820ffe1f-8e39-473f-81f2-6a18c045782a
opensearch-dashboards | {"type":"log","@timestamp":"2022-09-19T14:25:18Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}
The strange part is that the KID mismatch. Looking at the realms/master/protocol/openid-connect/certs
on my keycloak, I have two KIDs, neither matches the one from the opensearch log.
Log: 820ffe1f-8e39-473f-81f2-6a18c045782a
realms/master/protocol/openid-connect/certs has this content:
{
"keys": [
{
"kid": "d_Zb7sCBVl4B-4GTKprqN7-kyu4ZTzcGFVgZQ-Ze9aA",
"kty": "RSA",
"alg": "RSA-OAEP",
"use": "enc",
"n": "13f4HmVk4wu0CkWGYc1Y6sv2sqigDPvHNKMEzJFg-KnkSY1Fp0W-W2Ljv6sqGI15TXzXdTT9-NOzMSnrQBkobpWD2BKu-0kJk9MkXm8b92FUVISOtcy7M6XpyYMd3pCCogtiZQwSJlXXkkkV6D470xRO0ZPJE81XDmbHWKIKZjwVaxer0W2rZs7X1D7wTXonK0QczUOONovaLGOGnKRC1iEfH5aSH57gVD2l7ARRHgfwKBhECMIJ9gYcLP2UjhMYF7bBZx5F9Lqdt3gJ46egeHAJAiodW5lH7y0RDeYMWujX2AWjY8tLJ5WIs0_HHmW37bseXjBYhdA3J-4sI23bBQ",
"e": "AQAB",
"x5c": [
"MIICmzCCAYMCBgGDHXyI2TANBgkqhkiG9w0BAQsFADARMQ8wDQYDVQQDDAZtYXN0ZXIwHhcNMjIwOTA4MTQyMjI0WhcNMzIwOTA4MTQyNDA0WjARMQ8wDQYDVQQDDAZtYXN0ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDXd/geZWTjC7QKRYZhzVjqy/ayqKAM+8c0owTMkWD4qeRJjUWnRb5bYuO/qyoYjXlNfNd1NP3407MxKetAGShulYPYEq77SQmT0yRebxv3YVRUhI61zLszpenJgx3ekIKiC2JlDBImVdeSSRXoPjvTFE7Rk8kTzVcOZsdYogpmPBVrF6vRbatmztfUPvBNeicrRBzNQ442i9osY4acpELWIR8flpIfnuBUPaXsBFEeB/AoGEQIwgn2Bhws/ZSOExgXtsFnHkX0up23eAnjp6B4cAkCKh1bmUfvLREN5gxa6NfYBaNjy0snlYizT8ceZbftux5eMFiF0Dcn7iwjbdsFAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAJJwXbIwfoe/n4prY1xukuo47/TPVJs7GgSOrVNFyggzn3aK/izJewqjFN3HSGzd4akZT9uyAhD5SjmnUxj1dD57i1KIqNC6p0EEavKv7lUgtHHFFB4mY6IW9qowmkqi2SWnycyCfzSNEYL8ypXRkIYuHmwSrf97UglCYt5GkFqpvRGxvf0h5ovOdDaMvTBFyzNdsCARb9pbOlyy27OGpTQL5+lcdpHSi52SSmb+r7wMrHOHH8CqnjhaGegRbq/Z5dSaqnC14EuiYEWbB9KGoiJgFmpSXsAEswJfgYKB3qTciMDgGunsOlayfi4Ek8M0Kfmj9Ms2OiRpS1taTVq731M="
],
"x5t": "boUDgQf9oSFgLg0vq4cafyj4qO0",
"x5t#S256": "woUHU8MSaU-ZXkTo2T9ZYrZ2u45TXweIf1X5g2ihtkA"
},
{
"kid": "o5Vns9YsZEh_jqDOprU2WFh22DHI0TPjDAr8DFnXTmk",
"kty": "RSA",
"alg": "RS256",
"use": "sig",
"n": "k0c-rWI78_wU3-aaTAFFJiYEEHbGGmJsi-Ph0G8EyKBO7-vKpuYgsJTcckJk0hbNyAz-_oiHFRrINH3PnTY-43sIJYE8qIGmLs6UDFxZCls4F7MV8MwYF7jCTXdK4mT-69jUpNwoXzf4ecVlvTQ7RMTcmDNaWckPO3fc3xR2oz3MAOC1yRur5bUfgWAGSg5vy2po2B_8ahnnhIkYPtncnmsX07ppf-HdHAhDwYKziOMT5UedjgGi8iVOZn5kBLz12e71qMK6f7iZOkc7-l2Vvc7Dngz5JcBZW5Yw4TulK-GsxYnNCFjEj7yizLRhI8NdzWxHHdO3uAZnE2veKrbvdQ",
"e": "AQAB",
"x5c": [
"MIICmzCCAYMCBgGDHXyH/DANBgkqhkiG9w0BAQsFADARMQ8wDQYDVQQDDAZtYXN0ZXIwHhcNMjIwOTA4MTQyMjI0WhcNMzIwOTA4MTQyNDA0WjARMQ8wDQYDVQQDDAZtYXN0ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCTRz6tYjvz/BTf5ppMAUUmJgQQdsYaYmyL4+HQbwTIoE7v68qm5iCwlNxyQmTSFs3IDP7+iIcVGsg0fc+dNj7jewglgTyogaYuzpQMXFkKWzgXsxXwzBgXuMJNd0riZP7r2NSk3ChfN/h5xWW9NDtExNyYM1pZyQ87d9zfFHajPcwA4LXJG6vltR+BYAZKDm/LamjYH/xqGeeEiRg+2dyeaxfTuml/4d0cCEPBgrOI4xPlR52OAaLyJU5mfmQEvPXZ7vWowrp/uJk6Rzv6XZW9zsOeDPklwFlbljDhO6Ur4azFic0IWMSPvKLMtGEjw13NbEcd07e4BmcTa94qtu91AgMBAAEwDQYJKoZIhvcNAQELBQADggEBADf1OxE7BgX4Z8hGpR30k7C4t4HnXlACZKU278i40p/RGUUE6P5QG8Ld0pwFq57vGAWvnDQ2BecZ9seq/oZiUcLct6mT2IBZIHPptGoxhRU+alC4gv6rR286pOJ2u5WKGAFZq2LAizEhZuFQihz7CITqY4Sy/DtTJXeYjnEqbh53a2Pn2PGW5GNwc+MrdQjpEYPsQ/sNtUlF+xoZx5LBfCpBcn8Gz3VBLzF/3TbgFtxL/gsF24ds4BipneoTNUvwtSqv8QquPlVJdLObzegaEgcFWuAPgAjCQ/6YbV7HaWdGK4fIo9WEUtxqgkgwYvp44XunBV8KM3/AuyVyjj3Nj+s="
],
"x5t": "c-ISSbU20QZjLIeEWehuGLE8znA",
"x5t#S256": "CW7M5wlNYGbBkkfkjVHekX90KZbJC_LazHg1IzBX0do"
}
]
}
So close… Any tips for this mismatch? Thanks!