Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch 2.14.0
Describe the issue:
As part of the TLS cert renewal process I want to use the REST API for hot reload of transport and http certificates. When reloading the already used certificates, everything works fine (response code 200). But when I want to apply newly generated, self signed certificates, I always run into the same exception telling me that New Certs do not have valid Issuer DN, Subject DN or SAN.
I’ve checked my configuration and the validity of the certs, everything seems to be fine here. If the application is restarted, the new certs are also picked up without any issue. Only reloading via the API produces this error.
Configuration:
Relevant part of opensearch.yml:
plugins.security.authcz.admin_dn:
- O=opensearch-x7ga,CN=admin
plugins.security.ssl.transport.pemcert_filepath: certificates/unit-transport.cert
plugins.security.ssl.transport.pemkey_filepath: certificates/unit-transport.key
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/root-ca.cert
plugins.security.ssl.http.pemcert_filepath: certificates/unit-http.cert
plugins.security.ssl.http.pemkey_filepath: certificates/unit-http.key
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/root-ca.cert
plugins.security.ssl.http.clientauth_mode: OPTIONAL
cluster.name: opensearch-x7ga
node.name: opensearch-30
node.roles:
- data
- ingest
- ml
- coordinating_only
- cluster_manager
discovery.seed_providers: file
plugins.security.disabled: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enforce_hostname_verification: true
plugins.security.restapi.roles_enabled:
- all_access
- security_rest_api_access
plugins.security.unsupported.restapi.allow_securityconfig_modification: true
plugins.security.ssl_cert_reload_enabled: true
Cert files:
ls -la certificates/
total 40
drwxrwx--- 2 snap_daemon root 4096 Jun 21 09:01 .
drwxrwx--- 11 snap_daemon root 4096 Jun 21 09:01 ..
-rw-r--r-- 1 snap_daemon root 1179 Jun 21 09:04 app-admin.cert
-rw-r--r-- 1 snap_daemon root 1704 Jun 21 09:04 app-admin.key
-rw-r--r-- 1 snap_daemon root 2343 Jun 21 09:04 chain.pem
-rw-r--r-- 1 snap_daemon root 1163 Jun 21 09:01 root-ca.cert
-rw-r--r-- 1 snap_daemon root 1349 Jun 21 09:04 unit-http.cert
-rw-r--r-- 1 snap_daemon root 1704 Jun 21 09:04 unit-http.key
-rw-r--r-- 1 snap_daemon root 1268 Jun 21 09:04 unit-transport.cert
-rw-r--r-- 1 snap_daemon root 1704 Jun 21 09:04 unit-transport.key
Subject Alternative Name in the certs include Registered ID:1.2.3.4.5.5 to make use of OIDs.
Relevant Logs or Screenshots:
API response:
{"error":{"root_cause":[{"type":"i_o_exception","reason":"OpenSearchSecurityException[Error while initializing http SSL layer from PEM: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.]; nested: Exception[New Certs do not have valid Issuer DN, Subject DN or SAN.];"}],"type":"i_o_exception","reason":"OpenSearchSecurityException[Error while initializing http SSL layer from PEM: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.]; nested: Exception[New Certs do not have valid Issuer DN, Subject DN or SAN.];","caused_by":{"type":"security_exception","reason":"Error while initializing http SSL layer from PEM: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.","caused_by":{"type":"exception","reason":"New Certs do not have valid Issuer DN, Subject DN or SAN."}}},"status":500}
Server logfile:
[2024-06-21T09:04:26,877][WARN ][r.suppressed ] [opensearch-30] path: /_plugins/_security/api/ssl/transport/reloadcerts, params: {certType=transport}
java.io.IOException: OpenSearchSecurityException[Error while initializing transport SSL layer from PEM: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.]; nested: Exception[New Certs do not have valid Issuer DN, Subject DN or SAN.];
at org.opensearch.security.dlic.rest.api.SecuritySSLCertsApiAction.reloadCertificates(SecuritySSLCertsApiAction.java:203) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.SecuritySSLCertsApiAction.lambda$securitySSLCertsRequestHandlers$3(SecuritySSLCertsApiAction.java:117) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.validation.ValidationResult.valid(ValidationResult.java:87) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.SecuritySSLCertsApiAction.lambda$securitySSLCertsRequestHandlers$5(SecuritySSLCertsApiAction.java:105) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.RequestHandler$RequestHandlersBuilder.lambda$add$2(RequestHandler.java:97) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.AbstractApiAction.lambda$prepareRequest$23(AbstractApiAction.java:614) [opensearch-security-2.14.0.0.jar:2.14.0.0]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) [?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [?:?]
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:854) [opensearch-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.opensearch.OpenSearchSecurityException: Error while initializing transport SSL layer from PEM: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:484) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.SecuritySSLCertsApiAction.reloadCertificates(SecuritySSLCertsApiAction.java:190) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
... 11 more
Caused by: java.lang.Exception: New Certs do not have valid Issuer DN, Subject DN or SAN.
at org.opensearch.security.ssl.DefaultSecurityKeyStore.validateNewCerts(DefaultSecurityKeyStore.java:644) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:462) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
at org.opensearch.security.dlic.rest.api.SecuritySSLCertsApiAction.reloadCertificates(SecuritySSLCertsApiAction.java:190) ~[opensearch-security-2.14.0.0.jar:2.14.0.0]
... 11 more
Any help would be highly appreciated!