Current certs are about to expire referenced in the OpenSearch yaml config. After I stage new certs in the same location, and doing a rolling restart, do I need anything else? Will the nodes talk to each other while others have new certs after a restart vs those with old certs? Do I need to run the securityadmin script? Nothing changed as far roles go.
Going to do coordinator, data and master nodes in this order.
Which method did you use to host your OpenSearch Clusters? (ex. helm, k8s operator, docker, etc)
Yes, you need .pem(s) or .crt(s) files to call securityadmin.sh when making a change in opensearch/config/opensearch-security directory. See the below code.
@stecino securityadmin.sh is required only for the security plugin configuration.
Certificates must be managed separately.
You’ll need to just replace certificates.
Which certificates expired? Node, client, RootCA or all of them?
How big is the cluster?
@pablo thanks for the reply. It’s the node certificate and one admin cert. I have total of 19 nodes, 6 coordinator, 3 master and 10 data nodes. So just replace those and do rolling restart or not even that?
@stecino You should also consider the hot reload. Since you’ll be restarting all of the nodes, you can enable hot reloading in opensearch.yml and avoid rolling restarts in the future.
For what I’ve understood, if plugins.security.ssl_cert_reload_enabled setting is true, superadmin can replace the expired certificates to new ones using the Reload Certificates API.
But what if @stecino 's cluster has been set neither plugins.security.ssl.certificates_hot_reload.enabled nor plugins.security.ssl_cert_reload_enabled? They seem like to be false by default.
In this case, just replacing certificates in the same file path is enough?
If we want to use the Reload Certificates API, do we have to set true to both certificates_hot_reload and ssl_cert_reload_enabled and restart all nodes?
No, it’s not. When hot reloading is not activate then rolling restart is required.
As per OpenSearch documentation. The below option enables active monitoring of the certificates keystore and replaces certificates once the keystore is updated.
Thanks for your time, then in my opinion it’s recommended to set those two options True, when users try to initially configure the clusters.
(I mean, there’s no need to leave those options to be default : False)
@pablo, question for you. What happens if my node DN is changing. How do I approach the problem? I tried to update one of my coordinator nodes with new certs, updated the DN in the config and after restart I am getting this. Rolled back and it’s working again.
[2025-09-08T18:03:16,260][WARN ][o.o.d.HandshakingTransportAddressConnector] [xxxxx-01-alpha] handshake failed for [connectToRemoteMasterNode[10.xx.x.xxx:9300]]
org.opensearch.transport.RemoteTransportException: [xxxxx-01-alpha][10.xx.x.xxx:9300][internal:transport/handshake]
Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported.
at org.opensearch.security.ssl.util.ExceptionUtils.createTransportClientNoLongerSupportedException(ExceptionUtils.java:68) ~[?:?]
at org.opensearch.security.transport.SecurityRequestHandler.messageReceivedDecorate(SecurityRequestHandler.java:292) ~[?:?]
Yes I did. New CN entry matches with the new certificate. As a result the node I made the change on doesn’t show up in my nodes list when I run _cart/nodes command
@stecino, As per our conversation, you can use regular expressions in plugins.security.authcz.nodes_dn instead of exact node’s DN. The nodes_dn must be updated in all nodes before replacing certificate so the existing nodes would recognize a new certificate from a joining node.
Let me know if you managed to replace your certificate.
Hello @pablo it was nice seeing you at the event!! I did update nodes_dn as we discussed on all the nodes with a rolling restart. After that I replaced the certificate, but still same handshake error.
@stecino It was great to meet you in person too. Do you see these errors on all of the nodes?
Did you also update plugins.security.ssl.transport.pemtrustedcas_filepath on all of the nodes?
Was the new certificate signed by a different RootCA?