How do I configure demo-replacement certificate for helm chart?

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2,5

Describe the issue:
I am using helm with a single custom node cert (for a 3-replica deployment) with demo certs disabled and get the following errors:
Transport client authentication no longer supported.
Demo certificates found (goes away if I set allow_unsafe_democertificates: false but If I use true with custom cert paths I get the transport error above

My question is: How do I configure a multi-node helm production deployment (WITHOUT demo certs)? (your answer would address how to generate the certs, and the opensearch yaml config - thanks!)

A further question is: How do I fix the transport error below, generate a custom node cert and configure helm for a multi-node deployment?

Please note: The Configuring TLS Certificates doc appears to address multi-node clusters but not Kubernetes statefulsets (for these, we need a dynamic solution, not hard-configured certs as in the doc)

My config appears to be incorrect - see the error below:

Transport client authentication no longer supported

I am using helm to deploy to Kubernetes.
I want to replace the demo certificate with a custom one.
I set allow_unsafe_democertificates: true # Ouch! See custom path below. How do I switch this off? It complains if true - demo certs found!
I set the mount path for my k8s secrets:

secretMounts:
  - name: opensearch-tls
    secretName: opensearch-tls
    path: /usr/share/opensearch/config/certs

I have generated the same set of certs for each of the keys for the demo cert, which are correctly mounted in /usr/share/opensearch/config/certs. In other words, I have the following config and I am able to list these within the running container, indicating that the certs are correctly mounted:

   pemcert_filepath: certs/esnode.pem
   pemkey_filepath: certs/esnode-key.pem
   pemtrustedcas_filepath: certs/root-ca.pem

My cert looks like this:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ==> REDACTED <==
        Signature Algorithm: ==> REDACTED <==
        Issuer: C===> REDACTED <==, ST===> REDACTED <==, L===> REDACTED <==, O===> REDACTED <==, OU===> REDACTED <==, CN=opensearch-cluster-master.<MY NAMESPACE>.svc.cluster.local
        Validity
            Not Before: Sep 28 10:56:00 2024 GMT
            Not After : Sep 28 10:56:00 2025 GMT
        Subject: C===> REDACTED <==, ST===> REDACTED <==, L===> REDACTED <==, O===> REDACTED <==, OU===> REDACTED <==, CN=opensearch-cluster-master.<MY NAMESPACE>.svc.cluster.local
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
==> REDACTED <==
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:opensearch-cluster-master.<MY NAMESPACE>.svc.cluster.local, DNS:localhost, IP Address:127.0.0.1, Registered ID:1.2.3.4.5.7 
            X509v3 Extended Key Usage: critical
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
==> REDACTED <==
            X509v3 Authority Key Identifier: 
==> REDACTED <==
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
==> REDACTED <==

See Registered ID: it has been modified
See CN: It has been modified
I am using RID which should mean that I do not have to configure distinguished names (DNs)

The setup works fine with the demo certs enabled. It is with the above config that it fails.

Any ideas, please?
If the cert above is incorret, what is the correct way to generate a single-cert configutation (in a multi-node setup for Kubernetes) ?

Can you please advise (or show me the docs) on how I should configure certs for a production deployment on Kubernetes (using the helm chart here ?

Thanks!

@virasana As per the OpenSearch helm chart’s documentation you need to disable the Demo configuration with DISABLE_INSTALL_DEMO_CONFIG variable.
This will stop the generation of demo certificates and allow_unsafe_democertificates:false won’t generate the mentioned error.

The helm chart supports a value which, I presume, sets the appropriate env var you have mentioned. I have set this value to false.

What about the transport error?

Many Thanks

@virasana The transport error regards the DN of the node’s certificate. Starting from version 2.0, transport client authentication/authorization has been deprecated.

You need to specify a list of nodes DNs in the opensearch.yml

  • plugins.security.nodes_dn (Static): Specifies a list of distinguished names (DNs) that denote the other nodes in the cluster. This setting supports wildcards and regular expressions. The list of DNs are also read from the security index in addition to the YAML configuration when plugins.security.nodes_dn_dynamic_config_enabled is true. If this setting is not configured correctly, the cluster will fail to form as the nodes will not be able to trust each other and will result in the following error: Transport client authentication no longer supported.

Try adding this option to config.plugins.security.nodes_dn in values.yml

Thank you Pablo. I will try this. I guess this implies a wildcard certificate?

@virasana You don’t have to use a wildcard certificate but you can use a regular expression to define similar DNs in nodes_dn.
Take a look at the example from OpenSearch docs.

Thank you Pablo.