SSL Error unable to configure Opensearch Cluster in 6 VMs

Security
,
1

Hi Team,

I have been trying to deploy Opemsearch in 6 VM’s (3 Master and 3 Data). I have used .rpm file in-order to deploy. But I couldn’t able to successfully configure a single node. I am using the SSL certs provided by our CA Server in the format of both .PEM and .PFX, and I have extracted the certs, root-ca and keys and whenever I am applying them in opensearch.yml file, it looks like it is not working, even after changing the cert types X509, PKCS8 & PKCS12 still not working and getting the below error -

[2022-08-27T08:59:14,490][WARN ][o.o.h.AbstractHttpServerTransport] [elknlr2cr2ms01.us.dell.com] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/10.175.173.85:9200, remoteAddress=/10.175.173.81:64232}
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 48454144202f20485454502f312e310d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a436f6e74656e742d547970653a206170706c69636174696f6e2f6a736f6e0d0a782d656c61737469632d70726f647563742d6f726967696e3a206c6f6773746173680d0a486f73743a20656c6b6e6c72326372326d7330312e75732e64656c6c2e636f6d3a393230300d0a557365722d4167656e743a204c6f6773746173682f382e312e3020284f533d4c696e75782d332e31302e302d313136302e33362e322e656c372e7838365f36342d616d6436343b204a564d3d45636c697073652041646f707469756d2d31312e302e313329206c6f6773746173682d6f75747075742d656c61737469637365617263682f31312e342e310d0a4163636570742d456e636f64696e673a20677a69702c6465666c6174650d0a417574686f72697a6174696f6e3a204261736963206247396e633352686332686663336c7a644756744f6d4631633349796333567764773d3d0d0a0d0a
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:480) ~[netty-codec-4.1.79.Final.jar:4.1.79.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[netty-codec-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:623) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:586) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [netty-transport-4.1.79.Final.jar:4.1.79.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.79.Final.jar:4.1.79.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.79.Final.jar:4.1.79.Final]
at java.lang.Thread.run(Thread.java:833) [?:?]

Please help me in configuring the cluster because it is bit important and urgent for us to use and explore opensearch, in-order to use it in prod environment at the earliest.

Regards,
Debashis

2

@dmallick19
Make sure your certificates are placed in the proper location in the filesystem - you should have them in the config directory, where your opensearch.yml file is, or in a subfolder below that.

Also, you should include the DN of your nodes certificates in the opensearch.yml file under:
plugins.security.nodes_dn:

Keep in mind that node certificates need both serverAuth and clientAuth in their EKU section.

3

HI @Mussorgsky ,

Thank you so much for your response, I have managed to fix the SSL issue but, there is a different error I am getting now as below -

[2022-08-29T10:26:14,913][WARN ][o.o.d.HandshakingTransportAddressConnector] [elknlr2cr2ms01.us.dell.com] handshake failed for [connectToRemoteMasterNode[10.175.173.86:9300]]
org.opensearch.transport.RemoteTransportException: [elknlr2cr2ms02.us.dell.com][10.175.173.86:9300][internal:transport/handshake]
Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported.
at org.opensearch.security.ssl.util.ExceptionUtils.createTransportClientNoLongerSupportedException(ExceptionUtils.java:63) ~[?:?]
at org.opensearch.security.transport.SecurityRequestHandler.messageReceivedDecorate(SecurityRequestHandler.java:265) ~[?:?]
at org.opensearch.security.ssl.transport.SecuritySSLRequestHandler.messageReceived(SecuritySSLRequestHandler.java:152) ~[?:?]
at org.opensearch.security.OpenSearchSecurityPlugin$7$1.messageReceived(OpenSearchSecurityPlugin.java:658) ~[?:?]
at org.opensearch.indexmanagement.rollup.interceptor.RollupInterceptor$interceptHandler$1.messageReceived(RollupInterceptor.kt:100) ~[?:?]
at org.opensearch.performanceanalyzer.transport.PerformanceAnalyzerTransportRequestHandler.messageReceived(PerformanceAnalyzerTransportRequestHandler.java:43) ~[?:?]
at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:106) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:249) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:132) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:114) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:769) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:175) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:150) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:115) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:94) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[?:?]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[?:?]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[?:?]

Please help me with this.

Thanks

4

@dmallick19
As per the release notes of version 2.0 the TransportClient authentication/authorization have been removed.

What application is trying to connect to your cluster via the Transport layer?
What is being executed when you get this error?

5

Hi @Mussorgsky ,

I am not trying to connect any application to the cluster, actually I am trying to form a new cluster with 3 master & 3 data nodes and after the all the configuration I have made got the me the above error. Below is my configuration. Please help me with this -

######## Start OpenSearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: elk-r2.pem
plugins.security.ssl.transport.pemkey_filepath: elk-r2.key
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: elk-r2.pem
plugins.security.ssl.http.pemkey_filepath: elk-r2.key
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:

  • “CN=elk-r2.dell.com,OU=DELLIT,O=Dell,L=AUSTIN,ST=TX,C=US”
    plugins.security.nodes_dn:
  • “CN=elknlr2cr2ms01.us.dell.com,OU=DELLIT,O=Dell”
  • “CN=elknlr2cr2ms02.us.dell.com,OU=DELLIT,O=Dell”
  • “CN=elknlr2cr2ms03.us.dell.com,OU=DELLIT,O=Dell”
    plugins.security.ssl.http.enabled_protocols:
  • “TLSv1.2”
  • “TLSv1.3”
    plugins.security.audit.type: internal_opensearch
    plugins.security.enable_snapshot_restore_privilege: true
    plugins.security.check_snapshot_restore_write_privileges: true
    plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]
    plugins.security.system_indices.enabled: true
    plugins.security.system_indices.indices: [“.plugins-ml-model”, “.plugins-ml-task”, “.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opensearch-notifications-”, “.opensearch-notebooks”, “.opensearch-observability”, “.opendistro-asynchronous-search-response*”, “.replication-metadata-store”]
    node.max_local_storage_nodes: 3

Also the nodes are the Subject Alternative Names from the certificate. We have one pfx and I have used openssl to extract root-ca, cert.pem and key.

Please let me know, how do I manage if TransportClient authentication/authorization is disabled.

Thanks

6

For the value of plugins.security.nodes_dn: please try using the DNs of the subject/owner of each node certificate (instead of the values in the SAN section)

1 Like
7

Hi @Mussorgsky ,

Could you please help me in finding the DN from the below cert -

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
61:00:01:0a:cc:78:c5:54:3a:9c:b7:de:2e:00:00:00:01:0a:cc
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = Texas, L = Round Rock, O = Dell Technologies, CN = Dell Technologies Issuing CA 101
Validity
Not Before: Aug 30 21:37:27 2022 GMT
Not After : Aug 30 21:47:27 2024 GMT
Subject: C = US, ST = TX, L = Round Rock, O = Dell, OU = DellIT, CN = elk-r2.dell.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
xxxxx
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Subject Alternative Name:
DNS:elknlr2cr2ms01.us.dell.com, DNS:elknlr2cr2ms02.us.dell.com, DNS:elknlr2cr2ms03.us.dell.com, DNS:elknlr2cr2dt01.us.dell.com, DNS:elknlr2cr2dt02.us.dell.com, DNS:elknlr2cr2dt03.us.dell.com, DNS:elknlr2cr2dt04.us.dell.com, DNS:elknlr2cr2dt05.us.dell.com, DNS:elknlr2cr2dt06.us.dell.com, DNS:elknlr2cr2kb01.us.dell.com, DNS:elknlr2cr2kb02.us.dell.com
X509v3 Subject Key Identifier:
E5:DC:8A:E6:9A:6B:97:C0:E3:E1:F8:3C:BE:61:34:D4:7B:54:D0:61
X509v3 Authority Key Identifier:
keyid:53:0C:03:50:71:73:FA:C4:EC:51:53:8C:45:2B:6D:CA:60:56:93:B6

        X509v3 CRL Distribution Points:
8

You’re interested in the “subject” value:

Then you just have to put it in the proper order:

CN=elk-r2.dell.com,OU=DellIT,O=Dell,L=Round Rock,ST=TX,C=US

9

@dmallick19 Have you got this solved? If not, could you try to use only CN in the nodes_dn as defined in your cert’s SAN?

plugins.security.nodes_dn:
  - "CN=elknlr2cr2ms01.us.dell.com"
  - "CN=elknlr2cr2ms02.us.dell.com"
  - "CN=elknlr2cr2ms03.us.dell.com"

You must place all the node names from the same cluster in nodes_dn in each opensearch.yml. If you miss any, it will be treated as a client connection and the Transport layer error will be produced.

10

Hi @pablo ,

I have tried with the given configuration and I am still getting the error below -

[2022-09-05T09:06:32,151][ERROR][o.o.s.t.SecurityRequestHandler] [elknlr2cr2ms01.us.dell.com] OpenSearchException[Transport client authentication no longer supported.]
[2022-09-05T09:06:32,280][ERROR][o.o.s.t.SecurityRequestHandler] [elknlr2cr2ms01.us.dell.com] OpenSearchException[Transport client authentication no longer supported.]
[2022-09-05T09:06:32,949][WARN ][o.o.d.HandshakingTransportAddressConnector] [elknlr2cr2ms01.us.dell.com] handshake failed for [connectToRemoteMasterNode[10.175.173.86:9300]]
org.opensearch.transport.RemoteTransportException: [elknlr2cr2ms02.us.dell.com][10.175.173.86:9300][internal:transport/handshake]
Caused by: org.opensearch.OpenSearchException: Transport client authentication no longer supported.
at org.opensearch.security.ssl.util.ExceptionUtils.createTransportClientNoLongerSupportedException(ExceptionUtils.java:63) ~[?:?]
at org.opensearch.security.transport.SecurityRequestHandler.messageReceivedDecorate(SecurityRequestHandler.java:265) ~[?:?]
at org.opensearch.security.ssl.transport.SecuritySSLRequestHandler.messageReceived(SecuritySSLRequestHandler.java:152) ~[?:?]
at org.opensearch.security.OpenSearchSecurityPlugin$7$1.messageReceived(OpenSearchSecurityPlugin.java:658) ~[?:?]
at org.opensearch.indexmanagement.rollup.interceptor.RollupInterceptor$interceptHandler$1.messageReceived(RollupInterceptor.kt:100) ~[?:?]
at org.opensearch.performanceanalyzer.transport.PerformanceAnalyzerTransportRequestHandler.messageReceived(PerformanceAnalyzerTransportRequestHandler.java:43) ~[?:?]
at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:106) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:249) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:132) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:114) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:769) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:175) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:150) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:115) ~[opensearch-2.2.0.jar:2.2.0]
at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:94) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[?:?]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[?:?]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:623) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:586) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]

And please find the opensearch.yml conf below -

######## Start OpenSearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: elk-r2.pem
plugins.security.ssl.transport.pemkey_filepath: elk-r2.key
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: elk-r2.pem
plugins.security.ssl.http.pemkey_filepath: elk-r2.key
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
Plugins.security.authcz.admin_dn:

- “CN=elk-r2.dell.com,OU=DellIT,O=Dell,L=Round Rock,ST=TX,C=US”

plugins.security.nodes_dn:

  • “CN=elknlr2cr2ms01.us.dell.com”
  • “CN=elknlr2cr2ms02.us.dell.com”
  • “CN=elknlr2cr2ms03.us.dell.com”
    plugins.security.audit.type: internal_opensearch
    plugins.security.enable_snapshot_restore_privilege: true
    plugins.security.check_snapshot_restore_write_privileges: true
    plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]
    plugins.security.system_indices.enabled: true
    plugins.security.system_indices.indices: [“.plugins-ml-model”, “.plugins-ml-task”, “.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opensearch-notifications-”, “.opensearch-notebooks”, “.opensearch-observability”, “.opendistro-asynchronous-search-response*”, “.replication-metadata-store”]
    node.max_local_storage_nodes: 3

Please help me in resolving this issue, its been long pending issue with me and I am struck at here and is a huge blocker for me.

Thanks

11

@dmallick19 since you seem to be using/sharing the same certificate for all your nodes, you may test with the following:

plugins.security.nodes_dn:
  - "CN=elk-r2.dell.com,OU=DellIT,O=Dell,L=Round Rock,ST=TX,C=US"

If you have other certificates, please add them to the nodes_dn array

The same values have to be present in the opensearch.yml file in ALL the nodes

(Also don’t forget to restart each node after making changes to opensearch.yml)

12

Hi @Mussorgsky ,

Thanks and sorry for the late response, the issue has been fixed now. Used wildcard value and it worked. But one issue I still have, that I am unable to by pass opensearch dashboard url using nginx proxy with ssl enabled. Please let me know, if you have any specific config to follow.

Thanks

13

HI @dmallick19 ,

I’m also facing the same issue while forming the cluster, please help to share the steps and config file. Thanks

14

Hi @ashokfredrick ,

You can use the configuration like the below -

plugins.security.ssl.transport.pemcert_filepath: elk-r2.pem
plugins.security.ssl.transport.pemkey_filepath: elk-r2.key
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: elk-r2.pem
plugins.security.ssl.http.pemkey_filepath: elk-r2.key
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:

  • “CN=elk-r2.dell.com,OU=DellIT,O=Dell,L=Round Rock,ST=TX,C=US”
    plugins.security.nodes_dn:
  • “CN=*,OU=DellIT,O=Dell,L=Round Rock,ST=TX,C=US”

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [“.plugins-ml-model”, “.plugins-ml-task”, “.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opensearch-notifications-”, “.opensearch-notebooks”, “.opensearch-observability”, “.opendistro-asynchronous-search-response*”, “.replication-metadata-store”]
node.max_local_storage_nodes: 3

15

Hi @dmallick19 ,

Thank you for the details.

I have tried based on the above details and it’s not working for me.

FYI, we are using the keystore and truststore , please advise.