OpenSearch version 2.6.0, Same issue faced with 2.5.0,1.3.5
SAML authenticating and failing without any exception. Logs attached.
Configuration:
config.yml
---
_meta:
type: "config"
config_version: 2
config:
dynamic:
http:
anonymous_auth_enabled: false
authc:
internal_auth:
order: 0
description: "HTTP basic authentication using the internal user database"
http_enabled: true
transport_enabled: true
http_authenticator:
type: "basic"
challenge: false
authentication_backend:
type: "internal"
saml_auth:
order: 1
description: "SAML provider"
http_enabled: true
transport_enabled: false
http_authenticator:
type: "saml"
challenge: true
config:
idp:
metadata_file: "metadata.xml"
entity_id: "<removed>"
sp:
entity_id: "<removed>"
forceAuthn: true
kibana_url: "https://kibana-url:5601"
subject_key: "FirstName"
roles_key: "Groups"
exchange_key: "<removed>"
authentication_backend:
type: "noop"
Dashboard.yml
timelion.ui.enabled: true
server.host: "0.0.0.0"
opensearch.hosts: https://<hostname>:9200
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
opensearch_security.auth.type: ["basicauth","saml"]
opensearch_security.auth.multiple_auth_enabled: true
server.xsrf.whitelist: [/_plugins/_security/saml/acs,/_opendistro/_security/saml/acs,/_plugins/_security/saml/acs/idpinitiated,/_opendistro/_security/saml/acs/idpinitiated,/_plugins/_security/saml/logout,/_opendistro/_security/saml/logout]
Relevant Logs:
[2023-03-08T12:56:43,144][DEBUG][o.o.s.a.BackendRegistry ] Check authdomain for rest noop/1 or 2 in total
[2023-03-08T12:56:43,146][DEBUG][o.o.s.a.BackendRegistry ] Rest user āUser [name=rj999@xyz.com, backend_roles=[ā¦groups eg:345245ā¦], requestedTenant=null]ā is authenticated
[2023-03-08T12:56:43,146][DEBUG][o.o.s.a.BackendRegistry ] securitytenant ānullā
[2023-03-08T12:56:43,672][DEBUG][o.o.s.a.BackendRegistry ] Check authdomain for rest internal/0 or 2 in total
[2023-03-08T12:56:43,672][DEBUG][o.o.s.a.BackendRegistry ] Rest user āUser [name=kibanaserver, backend_roles= , requestedTenant=null]ā is authenticated
Screenshot:
Login page redirect back to the login page after the saml auth. Attached screenshot.
pablo
March 9, 2023, 8:31pm
2
@Rajkumar Iāve tested your configuration and so far canāt find anything wrong.
I have only a question about the kibana_url
in config.yml. The URL is pointing to HTTPS but in opensearch_dashboards.yml you didnāt enable SSL and set OpenSearch Dashboards certificates.
Have you tried SAML Troubleshooting section in OpenSeach documentation?
What is your SAML IdP?
Did you get any other errors either in OpenSearch or OpenSearch Dashboards?
I assume that this case is a duplicate.
https://forum.opensearch.org/t/saml-login-not-working-after-authentication/13022
Could you close that case and continue troubleshooting here?
1 Like
@pablo Thanks for checking on this, we can close duplicate case.
Enabled SSL also in my configuration.
timelion.ui.enabled: true
server.host: 0.0.0.0
opensearch.hosts: ["https://<hostname>:9200"]
opensearch.ssl.verificationMode: none
opensearch.username: "kibanaserver"
opensearch.password: "kibanaserver"
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false
server.ssl.enabled: true
server.ssl.certificate: /cert.pem
server.ssl.key: /key.pem
opensearch.ssl.certificateAuthorities: /ca-cert.pem
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
opensearch_security.cookie.secure: true
opensearch_security.auth.type: ["basicauth","saml"]
opensearch_security.auth.multiple_auth_enabled: true
server.xsrf.whitelist: [/_plugins/_security/saml/acs,/_opendistro/_security/saml/acs,/_plugins/_security/saml/acs/idpinitiated,/_opendistro/_security/saml/acs/idpinitiated,/_plugins/_security/saml/logout,/_opendistro/_security/saml/logout]
After the authentication, the UI redirects to login page.
Rest user 'User [name=rj999@xyz.com, backend_roles=[..groups eg:345245..], requestedTenant=null]' is authenticated
> --> Working till here.
securitytenant 'null'
Check authdomain for rest internal/0 or 2 in total
Rest user 'User [name=kibanaserver, backend_roles=[], requestedTenant=null]' is authenticated
securitytenant 'null'
ā SAML IdP? IDMS based SAMP Auth is used within the organisation.
ā Share dashboard logs, seeing 302/304/401 responses.
GET /auth/saml/captureUrlFragment?nextUrl=%2F 200 12ms - 9.0B
GET /auth/saml/captureUrlFragment.js 200 1ms - 9.0B
GET /auth/saml/login?nextUrl=%2F&redirectHash=false 302 33ms - 9.0B
POST /_opendistro/_security/saml/acs 302 792ms - 9.0B
GET / 302 4ms - 9.0B
GET /app/login 200 17ms - 9.0B
GET /bootstrap.js 304 15ms - 9.0B
GET /ui/favicons/manifest.json 200 3ms - 9.0B
GET /ui/favicons/favicon-32x32.png 200 4ms - 9.0B
GET /node_modules/@osd/ui-framework/dist/kui_light.css 200 4ms - 9.0B
GET /translations/en.json 200 1ms - 9.0B
GET /ui/legacy_light_theme.css 200 5ms - 9.0B
GET /ui/fonts/inter_ui/Inter-UI-Regular.woff2 200 2ms - 9.0B
GET /api/v1/restapiinfo 401 2ms - 9.0B
GET /api/v1/configuration/account 401 1ms - 9.0B
POST /api/core/capabilities 200 302ms - 9.0B
GET /api/v1/multitenancy/tenant 401 1ms - 9.0B
GET /api/v1/auth/type 401 1ms - 9.0B
GET /api/v1/configuration/account 401 1ms - 9.0B
Thanks
Rajkumar
pablo
March 10, 2023, 4:32pm
4
@Rajkumar What is the exact redirect URL in the IDP?
@pablo
we have configured below 2 redirect URLs.
https://hostname:5601/_opendistro/_security/saml/acs/idpinitiated
https://hostname:5601/_opendistro/_security/saml/acs
pablo
March 13, 2023, 3:39pm
6
@Rajkumar Can you just keep https://hostname:5601/_opendistro/_security/saml/acs
for testing?
Have you tried SAML Troubleshooting guide from OpenSearch documentation?
It will give you extra DEBUG level information in OpenSearch logs.
@pablo
Have tested both the endpoints separately /_opendistro/_security/saml/acs
& /_opendistro/_security/saml/acs/idpinitiated
getting below 500 error.
{āstatusCodeā:500,āerrorā:āInternal Server Errorā,āmessageā:āInternal Errorā}
Have enabled DEBUG level and able to find below SAML logs. However the UI is redirecting back to the login page.
Rest user āUser [name=rj999@xyz.com, backend_roles=[ā¦groups eg:345245ā¦], requestedTenant=null]ā is authenticated
ā Working till here.
securitytenant ānullā
Check authdomain for rest internal/0 or 2 in total
Rest user āUser [name=kibanaserver, backend_roles= , requestedTenant=null]ā is authenticated
securitytenant ānullā
Thanks
Rajkumar
pablo
March 21, 2023, 5:01pm
8
@Rajkumar Could you share the role and role mapping config of the test user?
@pablo
Sharing the role & role mapping.
roles_mapping.yml
test-role:
hosts: []
users:
- "admin"
reserved: false
hidden: false
backend_roles:
- "345245" --> Mapped with one of groupids.
and_backend_roles: []
roles.yml
test-role:
reserved: false
hidden: false
cluster_permissions:
- "cluster_composite_ops"
- "indices_monitor"
index_permissions:
- index_patterns:
- "test*"
dls: ""
fls: []
masked_fields: []
allowed_actions:
- "read"
tenant_permissions:
- tenant_patterns:
- "test"
allowed_actions:
- "kibana_all_read"
static: false
Please let me know, if I am missing anything.
Thanks
Rajkumar
Hi @pablo
Looks like we are hitting with this issue.
We have many Roles IDs for the for the user. Ppl with less roles are working fine.
opened 08:01PM - 07 Oct 20 UTC
enhancement
triaged
OpenID integration stopped working after the upgarde of Kibana to version 1.10.1ā¦ .
Checking the headers I found that the `set-cookie` header is ignored due to the size limit.
Here is a message from the browser dev tools:
```Set-Cookie header is ignored in response from url: https://logs-test.us.dev.kube.company.com/auth/openid/login?state=abKJ_B2F8UGxIw7HWbhWIp&code=a950100f-19d1-4bb6-8b50-0557362be14b.1e86c530-392c-4bd2-81fa-ca31a25db5f2.208d33ca-1735-423c-8075-8860fa0a8c34. Cookie length should be less than or equal to 4096```
Can you please let us know tentative release date for below request #1352 ?
opensearch-project:main
ā jochen-kressin:cookie-payload-split
opened 03:43PM - 28 Feb 23 UTC
### Description
Second shot at a solution for #1311. This should also be seen aā¦ s more of a RFC than a final pull request. Also, at the moment this only covers SAML.
With this PR, we create additional cookies without changing the Dashboards core code. Unfortunately, this required some not very nice workarounds:
- Setting a cookie value and clearing a cookie is done via the raw hapi request's cookieAuth.set() method. The cookieAuth in turn comes from the hapi cookie plugin. The reason for doing this is that we don't have access to the response.state() method with which we'd normally set the cookie
https://github.com/opensearch-project/security-dashboards-plugin/pull/1352/files#diff-0558556cc54b27144ba1c7e6c85065a7c37563f400171438153a8faca97bd757R26
- While it is possible to write a cookie without registering it with the hapi server first, this led to some problems with the cookie not being properly cleared. Hence, we opted for another not so nice workaround: We access the SessionStorage object to access the hapi server, and there we add the cookies
https://github.com/opensearch-project/security-dashboards-plugin/pull/1352/files#diff-94c1c398c7b3a2cd568a7ac3bfcd72dc48428eb1995613276a013be4413338c6R60
#### Additional info
- ~For this PoC we have just added two extra cookies. However, the quantity of extra cookies could be made configurable.~
- ~At the moment, the cookies are not encoded using "Iron". This could be changed in the cookie options though. We'd just have to make sure that we account for the extra space required when using Iron when splitting up the value into the cookies.~
- ~The names for the cookies are hardcoded as well. They use the cookie.name from the config, and appends 'saml_1' and 'saml_2'. This should probably be made configurable.~
- The number of extra cookies to use is now configurable
- The name of the extra cookies are now configurable. The extra cookies will use the configuration value as a prefix, and the append _1, _2 and so on, depending on how many cookies are used
#### Regarding the additional cookies
The number of extra cookies is configurable now, and defaults to 3. When splitting up the value we take the iron encoding into account, which adds considerably to the written cookie's size
The logic for that can be found here: https://github.com/opensearch-project/security-dashboards-plugin/pull/1352/files#diff-0558556cc54b27144ba1c7e6c85065a7c37563f400171438153a8faca97bd757R82
- We assume that Iron adds around 50%
- Based on that we try to split up the value, using as few extra cookies as possible
- If we know, based on the length of the value to written and the number of configured extra cookies, that we won't have enough cookies to account for the 50% we try to write anyway, but we divide the value evenly between all available cookies. Based on the value, Iron sometimes adds less overhead.
- I'm not sure if this is the best way, so I would appreciate your input on this.
Another caveat with having the number of extra cookies configurable would be if the administrators decrease that number, then existing cookies would not be read correctly.
To mitigate this, I guess we could store the maximum number of cookies used in the `authHeaderValueExtra` when the cookies are written.
But - I would wait for your feedback first, maybe you'd prefer not having the number of cookies configurable at all?
I could also give making the number of cookies completely dynamic based on the needed length another try.
The issue I ran in with this was that I wasn't able to delete the cookies if I did not register them with the Hapi server first.
#### Iron cookie size
I ran some tests on how much Iron adds to the final cookie size.
This is highly depending on the string, though. In my tests I added different roles to the token, and then compared.
For the SAML provided by @nibix both the compression and the Iron overhead were much better.
Roles | Original JWT | Compressed | Final Cookie | Iron overhead, ca | Comment
------------|--------|--------|----------------|---|-----
5 | 733 | 764 | 1260 | +65% |
10 | 839 | 844 | 1367 | +62% |
50| 1679 | 1464 | 2199 | +50% |
100 | 2791 | 2264 | 3266 | +44% |
150 | 3923 | 3036 | !!! >4096 | | Too large for the browser
200 | 5065 | 3808 | !!! >4096 | | Too large for the browser
Test SAML payload | 14096 | 2860 | 4055 | +42% | Tested with the SAML test token from the issue
#### Still open
- [x] Test error handling if the unsplit value is missing/corrupted.
We added checks to the `isValidCookie` method that will invalidate the cookie if the value is corrupted somehow, or if writing the cookies did not work.
While this improves the situation a bit, the user will still run into the "too many redirects" error if they have an active session with the IdP, which will then just redirect back the user to Dashboards. If the token still can't be written the redirecting will commence...
- [x] ~The getCookie for SAML needs to be changed before writing the authorization header value https://github.com/opensearch-project/security-dashboards-plugin/pull/1352/files#diff-94c1c398c7b3a2cd568a7ac3bfcd72dc48428eb1995613276a013be4413338c6R118~
#### Untested
- [ ] Setting cookies after an authorization header was found. E.g. here: https://github.com/opensearch-project/security-dashboards-plugin/pull/1352/files#diff-d782daf5a30c26f91c0430ed18479194b9b562026646e440521516a3e4290651R173
- [x] The refresh token flow in Openid
### Category
Bug fix
### Why these changes are required?
Allow users with many roles to authenticate with SAML in Dashboards.
### What is the old behavior before changes and new behavior after changes?
Such users could not log in into Dashboards.
### Issues Resolved
- #1311
- #516
### Testing
- Unit testing
- We still need some sample payloads to verify the effectiveness
### Check List
- [x] New functionality includes testing
- [ ] New functionality has been documented
- [x] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
Thanks
Rajkumar