Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearchproject/opensearch-dashboards:2.7.0
Describe the issue:
I cannot get enough logging details to determine the root cause of the following http response:
{“statusCode”:401,“error”:“Unauthorized”,“message”:“Authentication Exception”}
Configuration:
In the opensearch-dashboards helm values.yaml file:
config:
opensearch_dashboards.yml:
opensearch_security.auth.type: “jwt”
opensearch_security.jwt.url_param: “jwt_token”
# removes "first certificate" validation error
# per https://github.com/opensearch-project/security-dashboards-plugin/issues/872
opensearch.ssl.verificationMode: none
opensearch.username: "admin"
opensearch.password: "admin"
opensearch.requestHeadersWhitelist: [ authorization,securitytenant ]
# per https://forum.opensearch.org/t/error-401-authorization-required/11589
opensearch_security.cookie.secure: false
opensearch_security.session.keepalive: false
# server.ssl.enabled: false
opensearch.logQueries: true
logging.verbose: true
In the opensearch helm values.yaml file:
opensearch.yml:
cluster.name: opensearch-cluster
# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0
logger.level: "DEBUG"
# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# Implicitly done if ".singleNode" is set to "true".
# discovery.type: single-node
# Start OpenSearch Security Demo Configuration
# WARNING: revise all the lines below before you go into production
plugins:
security:
ssl:
transport:
pemcert_filepath: esnode.pem
pemkey_filepath: esnode-key.pem
pemtrustedcas_filepath: root-ca.pem
enforce_hostname_verification: false
http:
enabled: true
pemcert_filepath: esnode.pem
pemkey_filepath: esnode-key.pem
pemtrustedcas_filepath: root-ca.pem
allow_unsafe_democertificates: true
allow_default_init_securityindex: true
authcz:
admin_dn:
- CN=kirk,OU=client,O=client,L=test,C=de
audit.type: internal_opensearch
enable_snapshot_restore_privilege: true
check_snapshot_restore_write_privileges: true
restapi:
roles_enabled: ["all_access", "security_rest_api_access"]
system_indices:
enabled: true
indices:
[
".opendistro-alerting-config",
".opendistro-alerting-alert*",
".opendistro-anomaly-results*",
".opendistro-anomaly-detector*",
".opendistro-anomaly-checkpoints",
".opendistro-anomaly-detection-state",
".opendistro-reports-*",
".opendistro-notifications-*",
".opendistro-notebooks",
".opendistro-asynchronous-search-response*",
]
######## End OpenSearch Security Demo Configuration ########
config.yml:
dynamic:
http:
anonymous_auth_enabled: false
authc:
# basic_internal_auth_domain:
# description: “Authenticate via HTTP Basic against internal users database”
# http_enabled: true
# transport_enabled: true
# order: 4
# http_authenticator:
# type: basic
# challenge: true
# authentication_backend:
# type: intern
jwt_auth_domain:
description: “Authenticate via Json Web Token”
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: jwt
challenge: false
config:
signing_key: |
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAl5sENm/wwncxJ6RwPOjb\nY7aE/X6h3aJi5XdFdm4XUOx7dc9qqhkNctX8js1AkqFnm1NvVETE3q1QpoCUd03z\nmKAz167GkFwkrvn6Khe/aWANWSLfMlnnpTCoZbCqesrNtIbROrLuDgwOwENHpAHS\n016KKR+pBjsqbay/ITCJSlzyu7uK0bz/2GzvwIt/74yemYbYKsrDaBkA2Qf687CR\nK0Q0WKMIzQi8Ykdd6HBuY7GXLz2yB7yI84bGCzadv6Mv0qMLoW05jZxzinatDoiZ\nGAes6pWqF7JUaWpZJnqKFizFhF4RdNo2T0k4ouyxbsbZIT65RN5FhoSm226uqQVS\nUbGWH/9Hj27RLtzuw6zHsccvAYdTDWYiqJp7mVYe71jYSVGpM819QTlERLUWNcv3\nTUwEduKcIgBGPoW3yS03VaxkZNlLcM1yWGiwv8Ze613iWzovI6j/Vk5neJP0na+P\nG68bz95AQsYkLabCbaEzzWEUqxc8mkBON7/HRjNG50kLnLBt5xucgWQFCwFOtPWA\nROxic/BbgWYBVKo9dFZoWpO51AYBd5if/VQ6YesM/WNCcpTMkQKbtvVXjLGuPyzG\n8ZnYzXBrCmuvIhcOznbjvk6H3t5k8iwS+7Ro1+VS+Z9F8QdetW1Rj9BwKGIHo056\nZUsyNA2guF8TGg2w8BclEo0CAwEAAQ==
-----END PUBLIC KEY-----
# jwt_header: “Authorization”
jwt_url_parameter: “jwt_token”
jwt_clock_skew_tolerance_seconds: 30
roles_key: scope
subject_key: sub
authentication_backend:
type: noop
Relevant Logs or Screenshots:
In my typescript web app, I invoke opensearch-dashboards with the following code:
const url = http://localhost:5601/app/home?jwt_token=${token}
window.open(url, ‘_blank’, ‘noreferrer’)
A new browser tab opens at the above URL with the following:
{“statusCode”:401,“error”:“Unauthorized”,“message”:“Authentication Exception”}
When I look at the the pod logs for opensearch-dashboards pod, I only see the following related logs:
{“type”:“log”,“@timestamp”:“2023-06-05T16:42:46Z”,“tags”:[“debug”,“metrics”],“pid”:459,“message”:“Refreshing metrics”}
{“type”:“log”,“@timestamp”:“2023-06-05T16:42:46Z”,“tags”:[“debug”,“opensearch”,“opendistro_security”,“query”],“pid”:459,“message”:“401\nGET /_plugins/_security/authinfo\n”}
{“type”:“response”,“@timestamp”:“2023-06-05T16:42:43Z”,“tags”:,“pid”:459,“method”:“get”,“statusCode”:401,“req”:{“url”:“/app/home?jwt_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcnBfdXNlciIsIm5hbWUiOiJzcnBfdXNlciIsImV4cCI6MTY4NTk3OTI3MSwiaWF0IjoxNjg1OTc1NjcxLCJuYmYiOjE2ODU5NzU2NzEsInJvbGVzIjpbImFsbF9hY2Nlc3MiXX0.ddAxpvKnvPGzLEBFxULLhYYVwwcnQW9C-nr1bxNZkt4wqgETaYBgUGOW0MH1MKTPyEexn3qnLfM6h9-GT-yVnBY1H2RoGSmDDjrG7PUZfuerxHUmYe1j-pUF-dArGmAz0krVJcDaaxC0ewrv1ShBccO41oSz8eObPRAGgGp8BEio_jeR8YyaZFvLcoKQz9E8Ns_YJVNTqDApKQMjmM9YMlx6Gbs3KbwGqIIZP86Xqj_XpkotHWyY8VQZe5NLpPsMAHVpqRgagIVoPbTV5an5-4sAl_Wf5QpLaTYT8Ua6p32ZQK1BEou8RPkjyKpPOIfwf24tIBk0tU2r12qMaQLwrL-DTjTUzW5bL3k5AejLBUKzIX1qQSepH6x7BL9cZUFVzMlAWZqje5npSvlXJ4WIqG4qZ_1hzUqNg-xjGTvM-Ro16HfkHM4vi-jpbt23BhWm1WpqBtNf8KQGQoT1GrxBogu0WdS3uskdZ0cCDEtw_Xl5t_PPdRxUpBGWDW441FjWeawQs4IkfaIGTlyVJiKmjlXLDvABEkdmdqZNe8jhK48EIPFXvUunmYpEBv30lida39A8N3BW9NlOnlpXkU9bk3XCHPulbT0Tszd0ZcNJjXzDvbAhvVyALY4qu6g3sD2Rv0MCLken-14U2ApXAu9IdkGK6NfSLeNqAifMjH6Xfhw”,“method”:“get”,“headers”:{“host”:“localhost:5601”,“connection”:“keep-alive”,“sec-ch-ua”:“"Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"”,“sec-ch-ua-mobile”:“?0”,“sec-ch-ua-platform”:“"macOS"”,“upgrade-insecure-requests”:“1”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36”,“accept”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,“sec-fetch-site”:“same-site”,“sec-fetch-mode”:“navigate”,“sec-fetch-user”:“?1”,“sec-fetch-dest”:“document”,“accept-encoding”:“gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“127.0.0.1”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36”},“res”:{“statusCode”:401,“responseTime”:3504,“contentLength”:9},“message”:“GET /app/home?jwt_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcnBfdXNlciIsIm5hbWUiOiJzcnBfdXNlciIsImV4cCI6MTY4NTk3OTI3MSwiaWF0IjoxNjg1OTc1NjcxLCJuYmYiOjE2ODU5NzU2NzEsInJvbGVzIjpbImFsbF9hY2Nlc3MiXX0.ddAxpvKnvPGzLEBFxULLhYYVwwcnQW9C-nr1bxNZkt4wqgETaYBgUGOW0MH1MKTPyEexn3qnLfM6h9-GT-yVnBY1H2RoGSmDDjrG7PUZfuerxHUmYe1j-pUF-dArGmAz0krVJcDaaxC0ewrv1ShBccO41oSz8eObPRAGgGp8BEio_jeR8YyaZFvLcoKQz9E8Ns_YJVNTqDApKQMjmM9YMlx6Gbs3KbwGqIIZP86Xqj_XpkotHWyY8VQZe5NLpPsMAHVpqRgagIVoPbTV5an5-4sAl_Wf5QpLaTYT8Ua6p32ZQK1BEou8RPkjyKpPOIfwf24tIBk0tU2r12qMaQLwrL-DTjTUzW5bL3k5AejLBUKzIX1qQSepH6x7BL9cZUFVzMlAWZqje5npSvlXJ4WIqG4qZ_1hzUqNg-xjGTvM-Ro16HfkHM4vi-jpbt23BhWm1WpqBtNf8KQGQoT1GrxBogu0WdS3uskdZ0cCDEtw_Xl5t_PPdRxUpBGWDW441FjWeawQs4IkfaIGTlyVJiKmjlXLDvABEkdmdqZNe8jhK48EIPFXvUunmYpEBv30lida39A8N3BW9NlOnlpXkU9bk3XCHPulbT0Tszd0ZcNJjXzDvbAhvVyALY4qu6g3sD2Rv0MCLken-14U2ApXAu9IdkGK6NfSLeNqAifMjH6Xfhw 401 3504ms - 9.0B”}
{“type”:“log”,“@timestamp”:“2023-06-05T16:42:46Z”,“tags”:[“debug”,“http”,“server”,“OpenSearchDashboards”,“cookie-session-storage”],“pid”:459,“message”:“Error: Unauthorized”}
{“type”:“response”,“@timestamp”:“2023-06-05T16:42:46Z”,“tags”:,“pid”:459,“method”:“get”,“statusCode”:401,“req”:{“url”:“/favicon.ico”,“method”:“get”,“headers”:{“host”:“localhost:5601”,“connection”:“keep-alive”,“sec-ch-ua”:“"Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"”,“sec-ch-ua-mobile”:“?0”,“user-agent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36”,“sec-ch-ua-platform”:“"macOS"”,“accept”:“image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8”,“sec-fetch-site”:“same-origin”,“sec-fetch-mode”:“no-cors”,“sec-fetch-dest”:“image”,“referer”:“http://localhost:5601/app/home?jwt_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcnBfdXNlciIsIm5hbWUiOiJzcnBfdXNlciIsImV4cCI6MTY4NTk3OTI3MSwiaWF0IjoxNjg1OTc1NjcxLCJuYmYiOjE2ODU5NzU2NzEsInJvbGVzIjpbImFsbF9hY2Nlc3MiXX0.ddAxpvKnvPGzLEBFxULLhYYVwwcnQW9C-nr1bxNZkt4wqgETaYBgUGOW0MH1MKTPyEexn3qnLfM6h9-GT-yVnBY1H2RoGSmDDjrG7PUZfuerxHUmYe1j-pUF-dArGmAz0krVJcDaaxC0ewrv1ShBccO41oSz8eObPRAGgGp8BEio_jeR8YyaZFvLcoKQz9E8Ns_YJVNTqDApKQMjmM9YMlx6Gbs3KbwGqIIZP86Xqj_XpkotHWyY8VQZe5NLpPsMAHVpqRgagIVoPbTV5an5-4sAl_Wf5QpLaTYT8Ua6p32ZQK1BEou8RPkjyKpPOIfwf24tIBk0tU2r12qMaQLwrL-DTjTUzW5bL3k5AejLBUKzIX1qQSepH6x7BL9cZUFVzMlAWZqje5npSvlXJ4WIqG4qZ_1hzUqNg-xjGTvM-Ro16HfkHM4vi-jpbt23BhWm1WpqBtNf8KQGQoT1GrxBogu0WdS3uskdZ0cCDEtw_Xl5t_PPdRxUpBGWDW441FjWeawQs4IkfaIGTlyVJiKmjlXLDvABEkdmdqZNe8jhK48EIPFXvUunmYpEBv30lida39A8N3BW9NlOnlpXkU9bk3XCHPulbT0Tszd0ZcNJjXzDvbAhvVyALY4qu6g3sD2Rv0MCLken-14U2ApXAu9IdkGK6NfSLeNqAifMjH6Xfhw",“accept-encoding”:"gzip, deflate, br”,“accept-language”:“en-US,en;q=0.9”},“remoteAddress”:“127.0.0.1”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36”,“referer”:“http://localhost:5601/app/home?jwt_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcnBfdXNlciIsIm5hbWUiOiJzcnBfdXNlciIsImV4cCI6MTY4NTk3OTI3MSwiaWF0IjoxNjg1OTc1NjcxLCJuYmYiOjE2ODU5NzU2NzEsInJvbGVzIjpbImFsbF9hY2Nlc3MiXX0.ddAxpvKnvPGzLEBFxULLhYYVwwcnQW9C-nr1bxNZkt4wqgETaYBgUGOW0MH1MKTPyEexn3qnLfM6h9-GT-yVnBY1H2RoGSmDDjrG7PUZfuerxHUmYe1j-pUF-dArGmAz0krVJcDaaxC0ewrv1ShBccO41oSz8eObPRAGgGp8BEio_jeR8YyaZFvLcoKQz9E8Ns_YJVNTqDApKQMjmM9YMlx6Gbs3KbwGqIIZP86Xqj_XpkotHWyY8VQZe5NLpPsMAHVpqRgagIVoPbTV5an5-4sAl_Wf5QpLaTYT8Ua6p32ZQK1BEou8RPkjyKpPOIfwf24tIBk0tU2r12qMaQLwrL-DTjTUzW5bL3k5AejLBUKzIX1qQSepH6x7BL9cZUFVzMlAWZqje5npSvlXJ4WIqG4qZ_1hzUqNg-xjGTvM-Ro16HfkHM4vi-jpbt23BhWm1WpqBtNf8KQGQoT1GrxBogu0WdS3uskdZ0cCDEtw_Xl5t_PPdRxUpBGWDW441FjWeawQs4IkfaIGTlyVJiKmjlXLDvABEkdmdqZNe8jhK48EIPFXvUunmYpEBv30lida39A8N3BW9NlOnlpXkU9bk3XCHPulbT0Tszd0ZcNJjXzDvbAhvVyALY4qu6g3sD2Rv0MCLken-14U2ApXAu9IdkGK6NfSLeNqAifMjH6Xfhw"},“res”:{“statusCode”:401,“responseTime”:6,“contentLength”:9},“message”:"GET /favicon.ico 401 6ms - 9.0B”}
{“type”:“log”,“@timestamp”:“2023-06-05T16:42:47Z”,“tags”:[“debug”,“opensearch”,“data”,“query”],“pid”:459,“message”:“200\nGET /_nodes?filter_path=nodes..version%2Cnodes..http.publish_address%2Cnodes.*.ip”}
