Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.5.0 and 2.10.0
Describe the issue:
We are using proxy authentication with opensearch (OS) and opensearch dashboards (OSD) using x-forwarded-for, x-proxy-user and x-proxy-roles. For the most part this works well however, requests originating from the Dev Tools Console.
We have a gateway proxy in front of OSD that is providing the x-forwarded-for, x-proxy-user and x-proxy-roles headers. We have a man-in-the-middle gateway between OSD and OS in which we are able to log the headers being passed between the two systems. We have observed the traffic between the two when in the “Discover” tab (which works) and the “Dev Tools Console” (which does not). In neither case is the x-forwarded-for header sent between the two systems. However, in the case of Discover we see this log message log.trace("xff resolved {} to {}", remoteAddress, isa); whereas in the case of Dev Tools we see this log message log.trace("no xff done for {}", request.getClass());.
Additionally, even though the instructions on this page state to configure the x-forwarded-for header as a requestHeadersAllowlist, if we include the header in the list, proxy authentication does not seem to work at all. If we remove it, we get the behavior listed above.
Finally, if we reconfigure the entire system to use a custom header (ap-forwarded-for) it works as expected for all calls from OSD.
---
_meta:
type: "config"
config_version: 2
config:
dynamic:
# Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
# Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
# Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
#filtered_alias_mode: warn
do_not_fail_on_forbidden: true
#kibana:
# Kibana multitenancy
#multitenancy_enabled: true
#server_username: kibanaserver
#index: '.kibana'
http:
anonymous_auth_enabled: false
xff:
enabled: true
# internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
internalProxies: '.*' # trust all internal proxies, regex pattern
### The below update is needed if doing proxy authentication with Dev Tools / Console. The default
### `x-forwarded-for` works for the rest of dashboards but not Dev Tools / Console. Make sure to
### update Dashboards to forward this header and update the AddProxyHeaders filter to use this header.
# remoteIpHeader: 'ap-forwarded-for'
authc:
basic_internal_auth_domain:
description: "Authenticate via HTTP Basic against internal users database"
http_enabled: true
transport_enabled: true
order: 4
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
proxy_auth_domain:
description: "Authenticate via proxy"
http_enabled: true
transport_enabled: true
order: 0
http_authenticator:
type: proxy
challenge: false
config:
user_header: "x-proxy-user"
roles_header: "x-proxy-roles"
authentication_backend:
type: noop
clientcert_auth_domain:
description: "Authenticate via SSL client certificates"
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: clientcert
config:
username_attribute: cn #optional, if omitted DN becomes username
challenge: false
authentication_backend:
type: noop
I have extensively tested proxy authentication in my lab using x-forwarded-for, x-proxy-user and x-proxy-roles in all different scenarios, and I am not able to reproduce the same behaviour (note: proxy only in front of OSD and no proxy between OSD and OS)
If the x-forwarded-for is not in the header or/and is not in opensearch.requestHeadersAllowlist: the authentication fails with 401 as expected. Then the x-forwarded-for is configured and sent properly the UI and Dev Tools/Console behaves as per normal.
Could you test your set-up without the “man-in-the-middle gateway between OSD and OS” (with the x-forwarded-for in opensearch.requestHeadersAllowlist:)?