# Is X-Forwarded-For used when validating proxy traffic?

**URL:** <https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506>\
**Category:** Security\
**Created:** [July 24, 2024, 8:04pm UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506 "2024-07-24T20:04:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mark.boyd](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mark.boyd](https://forum.opensearch.org/u/mark.boyd)\
**Post date:** [July 24, 2024, 8:04pm UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506/1 "2024-07-24T20:04:29Z")

</div>

**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch 2.12.0  
OpenSearch Dashboards 2.12.0

**Describe the issue** :

I am confused about how, or if, the `X-Forwarded-For` header is used when validating that traffic from a proxy is trusted.

[The proxy documentation says](https://opensearch.org/docs/latest/security/authentication-backends/proxy/#enable-proxy-detection):

> To determine whether a request comes from a trusted internal proxy, the Security plugin compares the remote address of the HTTP request with the list of configured internal proxies. If the remote address is not in the list, the plugin treats the request like a client request.

Later on, when [describing the configuration of having a proxy in front of OpenSearch Dashboards, which is how we have things deployed, the documentation says](https://opensearch.org/docs/latest/security/authentication-backends/proxy/#opensearch-dashboards-proxy-authentication):

> In this case, the remote address of the HTTP call is the IP of OpenSearch Dashboards, because it sits directly in front of OpenSearch

So in either case, it seems like the value of `X-Forwarded-For` is not used to determine whether a request matches the trusted IPs set in `internalProxies`. Instead, the detected remote IP address from the request is checked for a match with `internalProxies` to determine if the traffic is allowed.

I have tested using a local Docker setup and found that I can set any value for `X-Forwarded-For` from my proxy and the traffic will succeed as long as the remote address matches the regex in `internalProxies`. I also enabled trace logging on my Docker OpenSearch nodes and the logs I’ve pasted below seem to confirm these findings.

In my testing, the only caveat I’ve found is that the `X-Forwarded-For` header **is required** for proxy traffic to succeed, otherwise you receive a `401` error.

So my understanding is that `X-Forwarded-For` is required for proxy authentication, but the actual value of the header does determine whether traffic is allowed. Is my understanding correct?

If my understanding is correct, then I am confused how or if the values in the `X-Forwarded-For` header are used at all for security? And if the `X-Forwarded-For` header values aren’t used to validate traffic, then why is it required?

**Configuration** :

`config.yml`:

```auto
---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      xff:
        enabled: true
        # trust IP addresses from Docker network
        internalProxies: '172\.19\.0.*'
        remoteIpHeader: "x-forwarded-for"
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      proxy_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: extended-proxy
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
            attr_header_prefix: "x-proxy-ext-"
        authentication_backend:
          type: noop

```

`opensearch_dashboards.yml`:

```auto
server.name: opensearchDashboards
server.host: "0.0.0.0"
opensearch.hosts: [https://localhost:9200]

opensearch.ssl.verificationMode: none
opensearch.username: admin
opensearch.password: admin

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.readonly_mode.roles: ["opensearch_dashboards_read_only"]

# Use this setting if you are running opensearch dashboards without https
opensearch_security.cookie.secure: false

data.search.usageTelemetry.enabled: false
opensearch.requestHeadersAllowlist: ["securitytenant","Authorization","x-forwarded-for","x-proxy-user","x-proxy-roles","x-proxy-ext-spaceids","x-proxy-ext-orgids"]

opensearch_security.auth.type: "proxy"
opensearch_security.proxycache.user_header: "x-proxy-user"
opensearch_security.proxycache.roles_header: "x-proxy-roles"

# disable the welcome screen to make e2e tests less flaky
home.disableWelcomeScreen: true
# disable the new theme modal to make e2e tests less flaky
home.disableNewThemeModal: true

```

`nginx.conf`, configuration for Nginx proxy :

```auto
events {
  worker_connections 1024;
}

http {
  server {
    listen 8080;

    location / {
      proxy_pass http://opensearch-dashboards:5601;
      proxy_set_header X-Forwarded-For 5.6.7.8;
      proxy_set_header x-proxy-user test;
      proxy_set_header x-proxy-roles admin;
    }
  }
}

```

**Relevant Logs or Screenshots** :

```auto
[TRACE] 2024-07-24 19:42:03.697 [opensearch[opensearch-node1][transport_worker][T#7]] RemoteIpDetector - originalRemoteAddr 172.19.0.2
[TRACE] 2024-07-24 19:42:03.697 [opensearch[opensearch-node1][transport_worker][T#7]] RemoteIpDetector - concatRemoteIpHeaderValue 5.6.7.8
[TRACE] 2024-07-24 19:42:03.697 [opensearch[opensearch-node1][transport_worker][T#7]] RemoteIpDetector - Incoming request /_plugins/_security/dashboardsinfo with originalRemoteAddr '172.19.0.2', originalRemoteHost='opensearch-dashboards.docker_opensearch-net', will be seen as newRemoteAddr='5.6.7.8'
[TRACE] 2024-07-24 19:42:03.697 [opensearch[opensearch-node1][transport_worker][T#7]] XFFResolver - xff resolved opensearch-dashboards.docker_opensearch-net/172.19.0.2:40364 to /5.6.7.8:40364

```

---

<div class="post-metadata">

**Author:** ![mark.boyd](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mark.boyd](https://forum.opensearch.org/u/mark.boyd)\
**Post date:** [July 24, 2024, 8:09pm UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506/2 "2024-07-24T20:09:29Z")

</div>

Also, [I’ve browsed the code that detects the remote IP](https://github.com/opensearch-project/security/blob/613aff1ea91f38ffb6942213e437cd50d881e46c/src/main/java/org/opensearch/security/http/RemoteIpDetector.java), and I can see that does read the `X-Forwarded-For` header and [**returns the last value from it** as the detected IP address](https://github.com/opensearch-project/security/blob/613aff1ea91f38ffb6942213e437cd50d881e46c/src/main/java/org/opensearch/security/http/RemoteIpDetector.java#L195), but that only confuses me more about why it doesn’t seem to matter when validating traffic.

[And the only integration test I can find for this behavior proves that if the **remote IP address does not match `internalProxies`** , then a `401` response is returned.](https://github.com/opensearch-project/security/blob/918c82155145a689c33fc1a7f5d4304876505628/src/integrationTest/java/org/opensearch/security/http/CommonProxyAuthenticationTests.java#L171-L181) . But there is no corresponding integration test proving that if the value of `X-Forwarded-For` does not match `internalProxies`, then the traffic is rejected.

---

<div class="post-metadata">

**Author:** ![Mantas](https://avatars.discourse-cdn.com/v4/letter/m/7bcc69/32.png) [@Mantas](https://forum.opensearch.org/u/Mantas)\
**Post date:** [August 13, 2024, 1:34pm UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506/3 "2024-08-13T13:34:16Z")

</div>

> [@mark.boyd](#):
>
> So my understanding is that `X-Forwarded-For` is required for proxy authentication, but the actual value of the header does determine whether traffic is allowed. Is my understanding correct?

Hi @mark.boyd,

I do believe your understanding is correct, the `X-Forwarded-For`, as I understand, is used to identify the source of the original requester, once the proxy allows it through, hence the `internalProxies: '172\.19\.0.*'`. Moreover, if I recall it correctly the `X-Forwarded-For`value can be any integer (not sure about string?) not necessary an IP (I will double check in my lab).

Let me know if you have any further questions or comments (or if I missed something).

Best,  
mj

---

<div class="post-metadata">

**Author:** ![mark.boyd](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mark.boyd](https://forum.opensearch.org/u/mark.boyd)\
**Post date:** [August 19, 2024, 2:51pm UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506/4 "2024-08-19T14:51:22Z")

</div>

@Mantas

Thanks for the reply! Unfortunately, I think my previous comment said the exact opposite of what I meant. What I should have said was:

My understanding is that the `X-Forwarded-For` header is required for proxy authentication, but the actual value of the header **does not determine whether traffic is allowed**. Is my understanding correct?

Based on our testing, I have concluded that the actual detected source IP address of the proxy server is what is used for comparison to the `internalProxies` configuration to determine if the traffic is allowed. This conclusion seems supported by the [RemoteIpDetector code in the security plugin](https://github.com/opensearch-project/security/blob/613aff1ea91f38ffb6942213e437cd50d881e46c/src/main/java/org/opensearch/security/http/RemoteIpDetector.java) and seemingly by this [integration test behavior](https://github.com/opensearch-project/security/blob/918c82155145a689c33fc1a7f5d4304876505628/src/integrationTest/java/org/opensearch/security/http/CommonProxyAuthenticationTests.java#L171-L181).

Does that sound correct?

Thanks again,  
Mark

---

<div class="post-metadata">

**Author:** ![Mantas](https://avatars.discourse-cdn.com/v4/letter/m/7bcc69/32.png) [@Mantas](https://forum.opensearch.org/u/Mantas)\
**Post date:** [August 26, 2024, 9:48am UTC](https://forum.opensearch.org/t/is-x-forwarded-for-used-when-validating-proxy-traffic/20506/5 "2024-08-26T09:48:30Z")

</div>

> [@mark.boyd](#):
>
> the actual detected source IP address of the proxy server is what is used for comparison to the `internalProxies` configuration to determine if the traffic is allowed.

@mark.boyd That is correct (based on my tests and understanding).

Best,  
mj
