Issue configuring Auth0 SAML

Security
1

Hi, I am trying to configure SAML with opendistro elasticsearch and Auth0 is our idp. After configuring the security.xml with recommended settings communication from kibana to elasticsearch is getting lost. If anyone have successfully configured SAML integration with opendistro, could you guide?

Thanks,
Kranthi

2

Welcome to the forum Kranthi

Can you please post the configuration file(s)?

Regards,
Clifford

3

Hi Clifford,
Thank you for the response. Here are the configuration file and log that I see in Kibana.
log [03:36:30.451] [info][plugins-service] Plugin “telemetryManagementSection” has been disabled since the following direct or transitive dependencies are missing or disabled: [telemetry]
log [03:36:30.483] [info][plugins-service] Plugin “newsfeed” is disabled.
log [03:36:30.484] [info][plugins-service] Plugin “telemetry” is disabled.
log [03:36:30.484] [info][plugins-service] Plugin “visTypeXy” is disabled.
log [03:36:30.901] [info][plugins-system] Setting up [46] plugins: [opendistroAlertingKibana,usageCollection,telemetryCollectionManager,kibanaUsageCollection,kibanaLegacy,securityOss,mapsLegacy,share,esUiShared,legacyExport,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,opendistroAnomalyDetectionKibana,opendistroIndexManagementKibana,opendistroSecurityKibana,opendistroReportsKibana,opendistroQueryWorkbenchKibana,embeddable,dashboard,opendistroNotebooksKibana,visualizations,visTypeVega,visTypeMarkdown,visTypeTimelion,timelion,visTypeTable,tileMap,regionMap,inputControlVis,opendistroGanttChartKibana,visualize,charts,visTypeTimeseries,visTypeVislib,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,bfetch]
log [03:36:34.745] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations…
log [03:36:39.340] [error][data][elasticsearch] [ResponseError]: Response Error
log [03:36:39.350] [error][savedobjects-service] Unable to retrieve version information from Elasticsearch nodes.
log [03:36:40.861] [error][data][elasticsearch] [ResponseError]: Response Error
log [03:36:43.364] [error][data][elasticsearch] [ResponseError]: Response Error
log [03:36:45.870] [error][data][elasticsearch] [ResponseError]: Response Error

Kibana Configuration

elasticsearch.hosts: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: [“securitytenant”,“Authorization”]
opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: [“Private”, “Global”]
opendistro_security.readonly_mode.roles: [“kibana_read_only”]

opendistro_security.cookie.secure: false
newsfeed.enabled: false
telemetry.optIn: false
telemetry.enabled: false
security.showInsecureClusterWarning: false

4

#Config.yml excerpt for SAML configuration
saml_auth_domain:
http_enabled: true
transport_enabled: false
order: 1
http_authenticator:
type: saml
challenge: true
config:
idp:
enable_ssl: false
verify_hostnames: false
metadata_file: elysiumanalytics_us_auth0_com-metadata.xml
entity_id: urn:elysiumanalytics.us.auth0.com
sp:
entity_id: ooL7fQ3MXyy2GazmtdI3E8aYPyNnBIoy
kibana_url: http://localhost:5601/
subject_key: http: //schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier
roles_key: # as forum not allowing me to put more than 2 links not placing actual role key
exchange_key: ‘tG0HPMzDMdwXRbhP4i1r7k9vvaqfMLAR’
authentication_backend:
type: noop

Elastic log excerpt printed while elasticserver is booting

[2021-03-02T09:02:33,434][INFO ][c.a.o.s.OpenDistroSecurityPlugin] [LAPTOP-UVPODTVK] 3 Open Distro Security modules loaded so far: [Module [type=MULTITENANCY, implementing class=com.amazon.opendistroforelasticsearch.security.configuration.PrivilegesInterceptorImpl], Module [type=REST_MANAGEMENT_API, implementing class=com.amazon.opendistroforelasticsearch.security.dlic.rest.api.OpenDistroSecurityRestApiActions], Module [type=AUDITLOG, implementing class=com.amazon.opendistroforelasticsearch.security.auditlog.impl.AuditLogImpl]]
[2021-03-02T09:02:33,435][INFO ][c.a.o.s.c.ConfigurationRepository] [LAPTOP-UVPODTVK] Background init thread started. Install default config?: true
[2021-03-02T09:02:34,126][INFO ][c.a.o.s.c.ConfigurationRepository] [LAPTOP-UVPODTVK] Index .opendistro_security already exists
[2021-03-02T09:02:34,207][INFO ][c.a.o.s.s.ConfigHelper ] [LAPTOP-UVPODTVK] Will update ‘audit’ with C:\Kranthi\elastic\ofde\opendistroforelasticsearch-1.12.0\plugins/opendistro_security/securityconfig/audit.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2021-03-02T09:02:38,761][INFO ][o.e.c.r.a.AllocationService] [LAPTOP-UVPODTVK] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.opendistro_security][0]]]).
[2021-03-02T09:02:39,270][INFO ][c.a.o.s.s.ConfigHelper ] [LAPTOP-UVPODTVK] Index .opendistro_security already contains doc with id audit, skipping update.
[2021-03-02T09:02:41,659][INFO ][o.o.c.c.InitializationService] [LAPTOP-UVPODTVK] Initializing OpenSAML using the Java Services API
[2021-03-02T09:02:59,056][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [LAPTOP-UVPODTVK] Metadata Resolver SamlFilesystemMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: New metadata successfully loaded for ‘C:\Kranthi\elastic\ofde\opendistroforelasticsearch-1.12.0\config\elysiumanalytics_us_auth0_com-metadata.xml’
[2021-03-02T09:02:59,277][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [LAPTOP-UVPODTVK] Metadata Resolver SamlFilesystemMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Next refresh cycle for metadata provider ‘C:\Kranthi\elastic\ofde\opendistroforelasticsearch-1.12.0\config\elysiumanalytics_us_auth0_com-metadata.xml’ will occur on ‘2021-03-02T06:32:58.622Z’ (‘2021-03-02T12:02:58.622+05:30’ local time)
[2021-03-02T09:03:01,220][INFO ][stdout ] [LAPTOP-UVPODTVK] [FINE] No subscribers registered for event class com.amazon.opendistroforelasticsearch.security.securityconf.DynamicConfigFactory$NodesDnModelImpl
[2021-03-02T09:03:01,223][INFO ][stdout ] [LAPTOP-UVPODTVK] [FINE] No subscribers registered for event class org.greenrobot.eventbus.NoSubscriberEvent
[2021-03-02T09:03:01,223][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Auditing on REST API is enabled.
[2021-03-02T09:03:01,224][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from REST API auditing.
[2021-03-02T09:03:01,226][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Auditing on Transport API is enabled.
[2021-03-02T09:03:01,227][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from Transport API auditing.
[2021-03-02T09:03:01,227][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Auditing of request body is enabled.
[2021-03-02T09:03:01,228][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Bulk requests resolution is disabled during request auditing.
[2021-03-02T09:03:01,229][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Index resolution is enabled during request auditing.
[2021-03-02T09:03:01,229][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Sensitive headers auditing is enabled.
[2021-03-02T09:03:01,230][INFO ][c.a.o.s.a.i.AuditLogImpl ] [LAPTOP-UVPODTVK] Auditing requests from kibanaserver users is disabled.
[2021-03-02T09:03:01,324][WARN ][c.a.o.s.a.r.AuditMessageRouter] [LAPTOP-UVPODTVK] No endpoint configured for categories [BAD_HEADERS, FAILED_LOGIN, MISSING_PRIVILEGES, GRANTED_PRIVILEGES, OPENDISTRO_SECURITY_INDEX_ATTEMPT, SSL_EXCEPTION, AUTHENTICATED, INDEX_EVENT, COMPLIANCE_DOC_READ, COMPLIANCE_DOC_WRITE, COMPLIANCE_EXTERNAL_CONFIG, COMPLIANCE_INTERNAL_CONFIG_READ, COMPLIANCE_INTERNAL_CONFIG_WRITE], using default endpoint

5

Hi @gkk306 Based on your logs, it would appear that kibana is not able to connect to elasticsearch. Did you try without any saml (just basic_auth). Does that work as expected?
Regarding SAML it would appear that you have not instructed kibana to use SAML (also need to whitelist the urls), see line below in kibana.yml file:

opendistro_security.auth.type: "saml"
server.xsrf.whitelist: ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]
1 Like
6

Hi Anthony,
without SAML configuration the communication is happening fine between elasticsearch and kibana. As soon as I configure it to use SAML the problem is occurring. Communication between kibana and elasticsearch is failing. I have added above mentioned lines already in the kibana configuration. While pasting content that might got trimmed. What could cause the communication to fail?

Thanks,
Kranthi

7

Hi @gkk306 couple of question:
What version of odfe are you using? (So I could try from my end)
How are you running odfe (docker, helm, RPM)
Is there any error messages in kibana (logs/UI) or elasticsearch when you try to access kibana?
I’ll try to reproduce when I have these

8

Hi Anthony,
Thank you for your response and following is the information you have asked.
ofde version: 1.12.0
Using the .tar.gz version uncompressed. Ran on centos and windows operating systems. Both the places getting same error.
All the logs elasticsearch and kibana related pasted part of the post. If you want me to provide specific log let me know.

Thanks,
Kranthi

9

@gkk306 could you add the below lines to kibana.yml

server.name: kibana
server.host: "0"

(If these lines were already included in kibana.yml please post full kibana.yml file)
If the error still persists, can you please confirm at what stage do you see error in the GUI, do you get to Auth0 login?

Please also make sure that the " are not “ in the kibana.yml, I know when you paste here they get changed, so just making sure. Should use ‘Preformatted text’ option for code snippets.

10

Hi Antony,
The elastic server starts fine and kibana doesn’t start at all. I see below errors when I try to start the kibana. I haven’t gone to the authentication page yet, it fails to start.

log [00:27:39.159] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations…
log [00:27:39.528] [error][data][elasticsearch] [ResponseError]: Response Error
log [00:27:39.543] [error][savedobjects-service] Unable to retrieve version information from Elasticsearch nodes.
log [00:27:41.688] [error][data][elasticsearch] [ResponseError]: Response Error
log [00:27:44.196] [error][data][elasticsearch] [ResponseError]: Response Error

When I have the server.host: “0” option on kibana.yml getting following error and server stops.
log [00:33:08.002] [fatal][root] { Error: getaddrinfo ENOTFOUND 0
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
errno: ‘ENOTFOUND’,
code: ‘ENOTFOUND’,
syscall: ‘getaddrinfo’,
hostname: ‘0’ }

FATAL Error: getaddrinfo ENOTFOUND 0

Here is kibana.yml content

server.name: kibana
#server.host: "0"

elasticsearch.hosts: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]

# Use this setting if you are running kibana without https
opendistro_security.cookie.secure: false

newsfeed.enabled: false
telemetry.optIn: false
telemetry.enabled: false
security.showInsecureClusterWarning: false

opendistro_security.auth.type: "saml"
server.xsrf.whitelist: ["/_opendistro/_security/saml/acs/idpinitiated", "/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]
11

@gkk306 Unfortunately I am not able to reproduce your error, your set up works for me on ubuntu.
Can you confirm the below:

  1. After you added SAML configuration in config.yml file, you uploaded that config file to ES using ./securityadmin.sh (without errors).

  2. ES is currently running and you are able to query using below:

    curl --insecure -XGET "https://localhost:9200/_cluster/health?pretty" -uadmin:admin

  3. When you start kibana, the last output is the one below (and nothing after this)
    log [03:36:34.745] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations…
    log [03:36:39.340] [error][data][elasticsearch] [ResponseError]: Response Error
    log [03:36:39.350] [error][savedobjects-service] Unable to retrieve version information from Elasticsearch nodes.
    log [03:36:40.861] [error][data][elasticsearch] [ResponseError]: Response Error
    log [03:36:43.364] [error][data][elasticsearch] [ResponseError]: Response Error
    log [03:36:45.870] [error][data][elasticsearch] [ResponseError]: Response Error

12

Hi Antony,
1) Yes, I have executed ./securityadmin.sh with suggested parameters as in documentation after SAML configuration changes in config.yml.
2) Following is the output (blank) when I use curl command to query the elasticsearch.
curl --insecure -X GET "https://localhost:9200/_cluster/health?pretty" -uadmin:admin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
3) This is after SAML changes at elasticsearch.When no SAML changes are introduced, I am able to start kibana fine.

Thanks,
Kranthi

13

@gkk306 if that curl command fails it would mean that the issue is not with kibana at all. Assuming again that curl command works fine without any saml config and you are able to retrieve the cluster health. Basic_auth should still be enabled in config.yml under “order 0” (since order 1 is saml in your case) - meaning authentication with ES using admin account should work unless there is a misconfiguration with config.yml file, which doesn’t always get reported during uploading process

14

Hi Antony,
Thank you for your inputs. As you are not able to replicate problem at your end can I request for a meeting where I can share screen and show you the environment and issue on my local. Please let me know what time works for you if possible.

Thanks,
Kranthi