"Transport Client Authentication no longer supported" error when deploying cluster with security plugin enabled

Hello,

I’m having trouble deploying an Opensearch 2.0.0 cluster with the security plugin enabled.
When deploying it in this way I get the following error:

[2022-06-02T16:21:38,726][WARN ][o.o.d.HandshakingTransportAddressConnector] [DNS] handshake failed for [connectToRemoteMasterNode[IP:9300]] org.opensearch.transport.RemoteTransportException: [DNS][IP:9300][internal:transport/handshake] Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported. at org.opensearch.security.ssl.util.ExceptionUtils.createTransportClientNoLongerSupportedException(ExceptionUtils.java:63) ~[?:?] at org.opensearch.security.transport.SecurityRequestHandler.messageReceivedDecorate(SecurityRequestHandler.java:270) ~[?:?] at org.opensearch.security.ssl.transport.SecuritySSLRequestHandler.messageReceived(SecuritySSLRequestHandler.java:153) ~[?:?] at org.opensearch.security.OpenSearchSecurityPlugin$7$1.messageReceived(OpenSearchSecurityPlugin.java:651) ~[?:?] at org.opensearch.indexmanagement.rollup.interceptor.RollupInterceptor$interceptHandler$1.messageReceived(RollupInterceptor.kt:118) ~[?:?] at org.opensearch.performanceanalyzer.transport.PerformanceAnalyzerTransportRequestHandler.messageReceived(PerformanceAnalyzerTransportRequestHandler.java:43) ~[?:?] at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:103) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:249) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:132) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:114) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:769) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:175) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:150) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:115) ~[opensearch-2.0.0.jar:2.0.0] at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:94) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?] at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1371) ~[?:?] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1234) ~[?:?] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1283) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:623) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:586) ~[?:?] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) ~[?:?] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[?:?] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?] at java.lang.Thread.run(Thread.java:833) [?:?] [2022-06-02T16:21:38,814][ERROR][o.o.s.t.SecurityRequestHandler] [DNS] OpenSearchException[Transport client authentication no longer supported.]

I have tried to find a way to disable transport client authentication but was not able to find any relevant information on the subject.
Any help provided would be greatly appreciated.

@vsgoncalo Please check this link.

I have the same issue, The answer from @pablo is not helpful for me I do not have an issue with securityadmin.sh my issue si that nides just do not form cluster. Nodes do not connect, when i dislable security it works as expected.
@vsgoncalo have you been able to solve this?

Hi @Marek1,

I believe I had a similar issue to yours and the answer from @pablo did not work for me either. I was eventually able to solve it. Unfortunately I’m not exactly sure what did the trick but I think that the problem was that I was using the absolute paths to the certificates in the configuration file whereas I should have been using the relative path to the config folder.

@vsgoncalo @pablo
Mine certs are directly in config dir and so my paths are only certs file names.
Like this:

plugins.security.disabled: false

plugins.security.ssl.transport.pemcert_filepath: cert.pem  
plugins.security.ssl.transport.pemkey_filepath : cert.key  
plugins.security.ssl.transport.pemtrustedcas_filepath: cet_ca.key

plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: cert.pem  
plugins.security.ssl.http.pemkey_filepath : cert.key  
plugins.security.ssl.http.pemtrustedcas_filepath: cet_ca.key

You have a typo on “plugins.security.ssl.http.enabked: true”. Could that be the cause of the error?
Also, I think you need to set the “plugins.security.nodes_dn” config as well.

Thansk for pointing out but it was just rewrite issue because my nodes are not connected to the internet and I can not copy-paste configs. Just question do I really need to run securityadmin.sh before i run OpenSearch every time I change opensearch,yml?

No need. I just launched each node with the correct configs and it ended up working.

@Marek1 @vsgoncalo The info provided previously, regards deprecation of Transport Client authentication/authorization. Port 9300 will be still in use for transport traffic.
This affects your deployment too. Once you disable demo certificates and you’ll use custom TLS certificates, you’ll need to add all your node’s certificates to plugins.security.nodes_dn in opensearch.yml file.

The error will appear if the node with TLS certificate, that is not in that list, will try to join the cluster.

@pablo Oke so I have the following config, but still errors like: Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported.
I also tried

  • transport.tcp.port: 9200
  • plugins.security.ssl.transport.enabled: “false”
  • plugins.security.nodes_dn:
    - ‘subject= CN=opensearch-cluster-master-headless.monitoring.svc.cluster.local,OU=DSH Cert Auth,O=DSH’
    • ‘subject= CN=opensearch-cluster-master-headless.monitoring.svc.cluster.local,OU=DSH Cert Auth,O=DSH’
    • ‘subject= CN=opensearch-cluster-master-headless.monitoring.svc.cluster.local,OU=DSH Cert Auth,O=DSH’
  • full paths like: plugins.security.ssl.http.pemtrustedcas_filepath: /usr/share/opensearch/config/ca_cert.pem

But no luck. Any ideas?

opensearch.yml: |
    cluster.name: opensearch-cluster

    network.host: "0.0.0.0"

    transport.host: "0.0.0.0"
    transport.tcp.port: 9300

    http.host: "0.0.0.0"
    http.port: 9200

    action.auto_create_index: true
    bootstrap.memory_lock: true


    plugins.security.authcz.admin_dn:
      - 'subject= CN=ADMIN,OU=DSH Cert Auth,O=DSH'
    plugins.security.nodes_dn:
      - 'subject= CN=opensearch,OU=DSH Cert Auth,O=DSH'
      - 'subject= CN=opensearch,OU=DSH Cert Auth,O=DSH'
      - 'subject= CN=opensearch,OU=DSH Cert Auth,O=DSH'

    plugins.security.allow_unsafe_democertificates: true
    plugins.security.ssl.http.enabled: "true"

    plugins.security.ssl.http.pemtrustedcas_filepath: ca_cert.pem
    plugins.security.ssl.http.pemcert_filepath: opensearch_cert.pem
    plugins.security.ssl.http.pemkey_filepath: opensearch_key.pem

    plugins.security.ssl.transport.enabled: "true"
    plugins.security.ssl.transport.enforce_hostname_verification: false
    plugins.security.ssl.transport.resolve_hostname: false

    plugins.security.ssl.transport.pemtrustedcas_filepath: ca_cert.pem
    plugins.security.ssl.transport.pemcert_filepath: opensearch_cert.pem
    plugins.security.ssl.transport.pemkey_filepath: opensearch_key.pem

    plugins.security.audit.type: internal_opensearch
    plugins.security.enable_snapshot_restore_privilege: true
    plugins.security.check_snapshot_restore_write_privileges: true

    plugins.security.allow_default_init_securityindex: true
    plugins.security.restapi.roles_enabled:
      ["all_access", "security_rest_api_access"]
    plugins.security.system_indices.enabled: true
    plugins.security.system_indices.indices:
      [
        ".opendistro-alerting-config",
        ".opendistro-alerting-alert*",
        ".opendistro-anomaly-results*",
        ".opendistro-anomaly-detector*",
        ".opendistro-anomaly-checkpoints",
        ".opendistro-anomaly-detection-state",
        ".opendistro-reports-*",
        ".opendistro-notifications-*",
        ".opendistro-notebooks",
        ".opendistro-asynchronous-search-response*",
        ".replication-metadata-store",
      ]

@Raki ‘subject=’ is not a part of the certificate DN.

Try the below instead.

    plugins.security.authcz.admin_dn:
      - 'CN=ADMIN,OU=DSH Cert Auth,O=DSH'
    plugins.security.nodes_dn:
      - 'CN=opensearch,OU=DSH Cert Auth,O=DSH'
      - 'CN=opensearch,OU=DSH Cert Auth,O=DSH'
      - 'CN=opensearch,OU=DSH Cert Auth,O=DSH'

that did it! tnx mate

@pablo I have to come back on that.
My master node does start up but gives error logs like:

[2022-07-01T10:54:10,856][ERROR][o.o.s.t.SecurityRequestHandler] [opensearch-cluster-master-0] OpenSearchException[Transport client authentication no longer supported.]
[2022-07-01T10:54:11,042][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-cluster-master-0] Not yet initialized (you may need to run securityadmin)
.[2022-07-01T10:54:11,536][ERROR][o.o.s.t.SecurityRequestHandler] [opensearch-cluster-master-0] OpenSearchException[Transport client authentication no longer supported.]

And my data nodes dont come up and they give the java error

[2022-07-01T10:45:12,759][WARN ][o.o.d.HandshakingTransportAddressConnector] [opensearch-cluster-data-0] handshake failed for [connectToRemoteMasterNode[10.42.71.118:9300]]
org.opensearch.transport.RemoteTransportException: [opensearch-cluster-master-1][10.42.71.118:9300][internal:transport/handshake]
Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported

And this is the verification
❯ openssl x509 -subject -nameopt RFC2253 -noout -in opensearch_cert_2022.pem subject=CN=opensearch,OU=DSH Cert Auth,O=DSH

@Raki Do you use the same certificate on all nodes? If so, you don’t need to duplicate them in the plugins.security.nodes_dn:

How do you deploy your nodes? Is it a service, docker or kube?

Try also escape spaces in the OU.

 plugins.security.authcz.admin_dn:
      - 'CN=ADMIN,OU=DSH\ Cert\ Auth,O=DSH'
    plugins.security.nodes_dn:
      - 'CN=opensearch,OU=DSH\ Cert\ Auth,O=DSH'

Got it to work in the end, thanks for all your suggestions.

@Raki Do you know what exactly solved your problem?

I made certs through the recommended way, We run 3 data nodes so I had to configure it like:

    plugins.security.authcz.admin_dn:
      - 'CN=admin,OU=DSH,O=DSH,ST=DH,C=NL'
    plugins.security.nodes_dn:
      - 'CN=opensearchall,OU=DSHCertAuth,O=DSH'
      - 'CN=opensearchall,OU=DSHCertAuth,O=DSH'
      - 'CN=opensearchall,OU=DSHCertAuth,O=DSH'

Also to skip any confusion I did not work with special characters nor spaces in the CN, OU, O

1 Like

Is this configuration in docker still good? It´s working on 1.3.3

cluster.name: openmaster-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false

plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: server-opensearch-cert.pem
plugins.security.ssl.http.pemkey_filepath: server-opensearch-key8.pem
plugins.security.ssl.http.pemtrustedcas_filepath: cacert-all.pem

plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de
  - CN=server-opensearch,OU=IT,O=Test,L=Prag,ST=Czech republic,C=CZ

    #path.repo: ["/mnt/snapshot"]

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########


plugins.security.ssl.http.enabled_protocols:
 - "TLSv1.2"
 - "TLSv1.3"