Opensearch with ldap

Hi Team,

I’m trying to enable opensearch with LDAP configuration. With security disabled mode, I’m able to access the opensearch endpoint, however with opensearch with LDAP, I’m getting below error.

Jul 6 08:30:26 os-opensearch-master-b5qwv-0 docker-entrypoint.sh[375]: {“type”: “server”, “timestamp”: “2021-07-06T08:30:26,213Z”, “level”: “ERROR”, “component”: “o.o.s.c.ConfigurationLoaderSecurity7”, “cluster.name”: “os”, “node.name”: “os-opensearch-master-b5qwv-0”, “message”: “Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)”, “cluster.uuid”: “fIRLyppiQySsjeZdWhpgmg”, “node.id”: “KhfQY8t6QdKxPRfeOT3vfA” }

While accessing the endpoint, it throws below error:

Command: curl -XGET https://localhost:30636 -u ‘admin:admin’ --insecure
Result: OpenSearch Security not initialized

Note: I deployed with one master and one data node
version: 1.0.0-rc1

config.yaml file is updated with valid authc and authz information.

Could you please help?

Hi @Kathiresan

Could you share your kibana.yml, elasticsearch.yml and config.yml files?

Hi @pablo Please find the opensearch.yaml file:

network.host: 0.0.0.0

cluster.name: osdemo

cluster.initial_master_nodes: [ “x.x.x.x” ]

node.master: true

######## Start OpenSearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: esnode.pem

plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem

plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem

plugins.security.ssl.transport.enforce_hostname_verification: false

plugins.security.ssl.http.enabled: true

plugins.security.ssl.http.pemcert_filepath: esnode.pem

plugins.security.ssl.http.pemkey_filepath: esnode-key.pem

plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem

plugins.security.allow_unsafe_democertificates: true

plugins.security.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch

plugins.security.enable_snapshot_restore_privilege: true

plugins.security.check_snapshot_restore_write_privileges: true

plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]

plugins.security.system_indices.enabled: true

plugins.security.system_indices.indices: [“.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opendistro-notifications-”, “.opendistro-notebooks”, “.opendistro-asynchronous-search-response*”]

node.max_local_storage_nodes: 3

######## End OpenSearch Security Demo Configuration ########

Opensearch.yaml contains specs related to elasticsearch

LDAP config as below:

_meta:

type: “config”

config_version: 2

config:

dynamic:

# Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index

# Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)

# Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently

#filtered_alias_mode: warn

#do_not_fail_on_forbidden: false

#kibana:

# Kibana multitenancy

#multitenancy_enabled: true

#server_username: kibanaserver

#index: '.kibana'

authc:

  kerberos_auth_domain:

    http_enabled: false

    transport_enabled: false

    order: 6

    http_authenticator:

      type: kerberos

      challenge: true

      config:

        # If true a lot of kerberos/security related debugging output will be logged to standard out

        krb_debug: false

        # If true then the realm will be stripped from the user name

        strip_realm_from_principal: true

    authentication_backend:

      type: noop

  basic_internal_auth_domain:

    description: "Authenticate via HTTP Basic against internal users database"

    http_enabled: true

    transport_enabled: true

    order: 4

    http_authenticator:

      type: basic

      challenge: true

    authentication_backend:

      type: intern

  proxy_auth_domain:

    description: "Authenticate via proxy"

    http_enabled: false

    transport_enabled: false

    order: 3

    http_authenticator:

      type: proxy

      challenge: false

      config:

        user_header: "x-proxy-user"

        roles_header: "x-proxy-roles"

    authentication_backend:

      type: noop

  jwt_auth_domain:

    description: "Authenticate via Json Web Token"

    http_enabled: false

    transport_enabled: false

    order: 0

    http_authenticator:

      type: jwt

      challenge: false

      config:

        signing_key: "base64 encoded HMAC key or public RSA/ECDSA pem key"

        jwt_header: "Authorization"

        jwt_url_parameter: null

        roles_key: null

        subject_key: null

    authentication_backend:

      type: noop

  clientcert_auth_domain:

    description: "Authenticate via SSL client certificates"

    http_enabled: false

    transport_enabled: false

    order: 2

    http_authenticator:

      type: clientcert

      config:

        username_attribute: cn #optional, if omitted DN becomes username

      challenge: false

    authentication_backend:

      type: noop

  ldap:

    description: "Authenticate via LDAP or Active Directory"

    http_enabled: false

    transport_enabled: false

    order: 5

    http_authenticator:

      type: basic

      challenge: false

    authentication_backend:

      # LDAP authentication backend (authenticate users against a LDAP or Active Directory)

      type: ldap

      config:

        # enable ldaps

        enable_ssl: false

        # enable start tls, enable_ssl should be false

        enable_start_tls: false

        # send client certificate

        enable_ssl_client_auth: false

        # verify ldap hostname

        verify_hostnames: true

        hosts:

        - x.x.x.x:port

        bind_dn: cn=Administrator,CN=Users,DC=sample,DC=example,DC=com

        password: password

        userbase: 'ou=people,dc=example,dc=com'

        # Filter to search for users (currently in the whole subtree beneath userbase)

        # {0} is substituted with the username

        usersearch: '(id={0})'

        # Use this attribute from the user as username (if not set then DN is used)

        username_attribute: null

authz:

  roles_from_myldap:

    description: "Authorize via LDAP or Active Directory"

    http_enabled: false

    transport_enabled: false

    authorization_backend:

      # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)

      type: ldap

      config:

        # enable ldaps

        enable_ssl: false

        # enable start tls, enable_ssl should be false

        enable_start_tls: false

        # send client certificate

        enable_ssl_client_auth: false

        # verify ldap hostname

        verify_hostnames: true

        hosts:

        - x.x.x.x:port

        bind_dn: cn=Administrator,CN=Users,DC=sample,DC=example,DC=com

        password: password

        rolebase: 'ou=groups,dc=example,dc=com'

        # Filter to search for roles (currently in the whole subtree beneath rolebase)

        # {0} is substituted with the DN of the user

        # {1} is substituted with the username

        # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute

        rolesearch: '(member={0})'

        # Specify the name of the attribute which value should be substituted with {2} above

        userroleattribute: null

        # Roles as an attribute of the user entry

        userrolename: disabled

        #userrolename: memberOf

        # The attribute in a role entry containing the name of that role, Default is "name".

        # Can also be "dn" to use the full DN as rolename.

        rolename: cn

        # Resolve nested roles transitive (roles which are members of other roles and so on ...)

        resolve_nested_roles: true

        userbase: 'ou=people,dc=example,dc=com'

        # Filter to search for users (currently in the whole subtree beneath userbase)

        # {0} is substituted with the username

        usersearch: '(id={0})'

        # Skip users matching a user name, a wildcard or a regex pattern

        #skip_users:

        #  - 'cn=Michael Jackson,ou*people,o=TEST'

        #  - '/\S*/'

Hi @Kathiresan

That config looks like default. LDAP is not enabled and not configured. Did you send the correct file?
You can DM the file if you don’t want to expose it on the forum.

Thanks Pablo. Do you have a slack channel or email through which I can share details?

Sharing errors messages observed after configuring ldap on behalf of Kath

Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: {“type”: “server”, “timestamp”: “2021-07-08T07:23:37,836Z”, “level”: “ERROR”, “component”: “o.o.s.s.t.SecuritySSLNettyTransport”, “cluster.name”: “ldap2”, “node.name”: “ldap2-opensearch-master-gzt6t-0”, “message”: “Exception during establishing a SSL connection: java.lang.IllegalStateException: transport not ready yet to handle incoming requests”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “stacktrace”: [“java.lang.IllegalStateException: transport not ready yet to handle incoming requests”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.TransportService.onRequestReceived(TransportService.java:974) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:177) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:120) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:102) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:713) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:155) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:130) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:95) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:87) [transport-netty4-client-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:620) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:583) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:37 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at java.lang.Thread.run(Thread.java:832) [?:?]”] }

Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: {“type”: “server”, “timestamp”: “2021-07-08T07:23:37,840Z”, “level”: “WARN”, “component”: “o.o.t.TcpTransport”, “cluster.name”: “ldap2”, “node.name”: “ldap2-opensearch-master-gzt6t-0”, “message”: “exception caught on transport layer [Netty4TcpChannel{localAddress=/10.192.1.18:9300, remoteAddress=/10.192.2.16:39924}], closing connection”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “stacktrace”: [“java.lang.IllegalStateException: transport not ready yet to handle incoming requests”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.TransportService.onRequestReceived(TransportService.java:974) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.handleRequest(InboundHandler.java:177) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.messageReceived(InboundHandler.java:120) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:102) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:713) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:155) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:130) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:95) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:87) [transport-netty4-client-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329) [netty-handler-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:620) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:583) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.59.Final.jar:4.1.59.Final]”,
Jul 8 07:23:38 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at java.lang.Thread.run(Thread.java:832) [?:?]”] }

=======================================================================================================================================
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: {“type”: “server”, “timestamp”: “2021-07-08T07:23:39,826Z”, “level”: “ERROR”, “component”: “o.o.s.c.ConfigurationLoaderSecurity7”, “cluster.name”: “ldap2”, “node.name”: “ldap2-opensearch-master-gzt6t-0”, “message”: “Exception while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)”, “cluster.uuid”: “USSqRtCPQemvlo8kcORlKw”, “node.id”: “kphcrQKZSlmMFhnPLRtiPw” ,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “stacktrace”: [“org.opensearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:203) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:189) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:72) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:53) ~[opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:192) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:136) [opensearch-index-management-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:99) [opensearch-performance-analyzer-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:234) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:154) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:190) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction.execute(TransportAction.java:168) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.action.support.TransportAction.execute(TransportAction.java:96) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:99) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:88) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:428) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.client.support.AbstractClient.multiGet(AbstractClient.java:546) [opensearch-1.0.0-rc1.jar:1.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.loadAsync(ConfigurationLoaderSecurity7.java:211) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.load(ConfigurationLoaderSecurity7.java:102) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationRepository.getConfigurationsFromIndex(ConfigurationRepository.java:375) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration0(ConfigurationRepository.java:321) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration(ConfigurationRepository.java:306) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at org.opensearch.security.configuration.ConfigurationRepository$1.run(ConfigurationRepository.java:166) [opensearch-security-1.0.0.0-rc1.jar:1.0.0.0-rc1]”,
Jul 8 07:23:39 ldap2-opensearch-master-gzt6t-0 docker-entrypoint.sh[1094]: “at java.lang.Thread.run(Thread.java:832) [?:?]”] }

Hi @rprakash

Just replied on DM. Your config.yml is missing basic authentication.