Set the level to debug and this is what i get in the ES log:
Sorry if it is too long and still had to remove some parts to be able to post it …
[2021-06-05T17:24:10,842][DEBUG][c.a.o.s.a.BackendRegistry] [node-1] Cannot authenticate rest user USER (or add roles) with authdomain internal/1 of [AuthDomain [backend=com.amazon.opendistroforelasticsearch.security.auth.internal.InternalAuthenticationBackend@6d387e9b, httpAuthenticator=com.amazon.opendistroforelasticsearch.security.http.HTTPBasicAuthenticator@645257a, order=1, challenge=true], AuthDomain [backend=com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend@624134f0, httpAuthenticator=com.amazon.opendistroforelasticsearch.security.http.HTTPBasicAuthenticator@63e84ebf, order=2, challenge=false]], try next
[2021-06-05T17:24:10,842][DEBUG][c.a.o.s.a.BackendRegistry] [node-1] Check authdomain for rest ldap/2 or 2 in total
[2021-06-05T17:24:10,852][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] verifyHostname true:
[2021-06-05T17:24:10,852][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] trustall false:
[2021-06-05T17:24:10,852][DEBUG][c.a.o.s.s.PemKeyReader ] [node-1] Path is is /etc/elasticsearch/certs/root-ca-digi.pem
[2021-06-05T17:24:10,853][DEBUG][c.a.o.s.s.PemKeyReader ] [node-1] Resolved /etc/elasticsearch/certs/root-ca-digi.pem to /etc/elasticsearch/certs/root-ca-digi.pem against /etc/elasticsearch
[2021-06-05T17:24:10,863][DEBUG][c.a.o.s.s.PemKeyReader ] [node-1] Path is is null
[2021-06-05T17:24:10,864][DEBUG][c.a.o.s.s.PemKeyReader ] [node-1] Path is is null
[2021-06-05T17:24:10,865][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] Use PEM to secure communication with LDAP server (client auth is false)
[2021-06-05T17:24:10,866][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] enabled ssl/tls protocols for ldaps [TLSv1.3, TLSv1.2]
[2021-06-05T17:24:10,866][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] Connect timeout: PT5S/ResponseTimeout: PT0S
[2021-06-05T17:24:10,866][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] bindDn cn=ldap_bind_account,dc=blabla,dc=com, password ****
[2021-06-05T17:24:10,866][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] Will perform simple bind with bind dn
[2021-06-05T17:24:10,982][DEBUG][o.l.s.AggregateTrustManager] [node-1] checkServerTrusted for sun.security.ssl.X509TrustManagerImpl@1a6fbf89 succeeded
[2021-06-05T17:24:10,982][DEBUG][o.l.s.AggregateTrustManager] [node-1] invoking getAcceptedIssuers for sun.security.ssl.X509TrustManagerImpl@1a6fbf89
[2021-06-05T17:24:11,040][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] Opened a connection, total count is now 1
[2021-06-05T17:24:11,051][DEBUG][o.l.SearchOperation ] [node-1] execute request=[org.ldaptive.SearchRequest@-1928768266::baseDn=ou=People,dc=blabla,dc=com, searchFilter=[org.ldaptive.SearchFilter@-1770077014::filter=(uid={0}), parameters={0=USER}], returnAttributes=[*, +], searchScope=SUBTREE, timeLimit=PT0S, sizeLimit=0, derefAliases=ALWAYS, typesOnly=false, binaryAttributes=null, sortBehavior=UNORDERED, searchEntryHandlers=null, searchReferenceHandlers=[org.ldaptive.referral.SearchReferralHandler$SearchReferenceHandler@7879e4ac], controls=null, referralHandler=org.ldaptive.referral.SearchReferralHandler@199b43ae, intermediateResponseHandlers=null] with connection=com.amazon.dlic.auth.ldap.backend.LDAPAuthorizationBackend$4@2ef098b5
[2021-06-05T17:24:11,066][DEBUG][o.l.p.j.NamingExceptionUtils] [node-1] naming exception class javax.naming.OperationNotSupportedException is ambiguous, maps to multiple result codes: [UNAVAILABLE_CRITICAL_EXTENSION, UNWILLING_TO_PERFORM]
[2021-06-05T17:24:11,068][DEBUG][c.a.d.a.l.b.LDAPAuthorizationBackend] [node-1] Closed a connection, total count is now 0
[2021-06-05T17:24:11,068][DEBUG][c.a.d.a.l.b.LDAPAuthenticationBackend] [node-1] Unable to authenticate user due to
org.ldaptive.LdapException: javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com'
at org.ldaptive.provider.ProviderUtils.throwOperationException(ProviderUtils.java:55) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.provider.jndi.JndiConnection.processNamingException(JndiConnection.java:619) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.provider.jndi.JndiConnection$JndiSearchIterator.initialize(JndiConnection.java:741) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.provider.jndi.JndiConnection.search(JndiConnection.java:463) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.SearchOperation.executeSearch(SearchOperation.java:103) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.SearchOperation.invoke(SearchOperation.java:85) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.SearchOperation.invoke(SearchOperation.java:15) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.AbstractOperation.execute(AbstractOperation.java:126) ~[ldaptive-1.2.3.jar:?]
at com.amazon.dlic.auth.ldap.util.LdapHelper$1.run(LdapHelper.java:67) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.dlic.auth.ldap.util.LdapHelper$1.run(LdapHelper.java:56) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]
at com.amazon.dlic.auth.ldap.util.LdapHelper.search(LdapHelper.java:56) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend.existsSearchingAllBases(LDAPAuthenticationBackend.java:259) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend.exists(LDAPAuthenticationBackend.java:212) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend.authenticate(LDAPAuthenticationBackend.java:89) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry$9.call(BackendRegistry.java:671) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry$9.call(BackendRegistry.java:664) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4875) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) [guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4870) [guava-25.1-jre.jar:?]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry.authcz(BackendRegistry.java:664) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:470) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.checkAndAuthenticateRequest(OpenDistroSecurityRestFilter.java:177) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.access$000(OpenDistroSecurityRestFilter.java:66) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter$1.handleRequest(OpenDistroSecurityRestFilter.java:113) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:258) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:340) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:191) [elasticsearch-7.10.2.jar:7.10.2]
at com.amazon.opendistroforelasticsearch.security.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:319) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:384) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:309) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:42) [transport-netty4-client-7.10.2.jar:7.10.2]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:28) [transport-netty4-client-7.10.2.jar:7.10.2]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.10.2.jar:7.10.2]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1518) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3332) ~[?:?]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3205) ~[?:?]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2996) ~[?:?]
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1875) ~[?:?]
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1798) ~[?:?]
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392) ~[?:?]
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358) ~[?:?]
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341) ~[?:?]
at org.ldaptive.provider.jndi.JndiConnection$JndiSearchIterator.search(JndiConnection.java:807) ~[ldaptive-1.2.3.jar:?]
at org.ldaptive.provider.jndi.JndiConnection$JndiSearchIterator.initialize(JndiConnection.java:735) ~[ldaptive-1.2.3.jar:?]
... 89 more
[2021-06-05T17:24:11,075][DEBUG][c.a.o.s.a.BackendRegistry] [node-1] Can not authenticate USER due to com.google.common.util.concurrent.UncheckedExecutionException: ElasticsearchSecurityException[[org.ldaptive.LdapException@1988524820::resultCode=UNWILLING_TO_PERFORM, matchedDn=null, responseControls=null, referralURLs=null, messageId=-1, message=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com', providerException=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com']]; nested: LdapException[javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com']; nested: OperationNotSupportedException[[LDAP: error code 53 - authentication required]];
com.google.common.util.concurrent.UncheckedExecutionException: ElasticsearchSecurityException[[org.ldaptive.LdapException@1988524820::resultCode=UNWILLING_TO_PERFORM, matchedDn=null, responseControls=null, referralURLs=null, messageId=-1, message=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com', providerException=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com']]; nested: LdapException[javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com']; nested: OperationNotSupportedException[[LDAP: error code 53 - authentication required]];
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) ~[guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-25.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4870) ~[guava-25.1-jre.jar:?]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry.authcz(BackendRegistry.java:664) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:470) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.checkAndAuthenticateRequest(OpenDistroSecurityRestFilter.java:177) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.access$000(OpenDistroSecurityRestFilter.java:66) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter$1.handleRequest(OpenDistroSecurityRestFilter.java:113) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:258) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:340) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:191) [elasticsearch-7.10.2.jar:7.10.2]
at com.amazon.opendistroforelasticsearch.security.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [opendistro_security-1.13.1.0.jar:1.13.1.0]
at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:319) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:384) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:309) [elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:42) [transport-netty4-client-7.10.2.jar:7.10.2]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:28) [transport-netty4-client-7.10.2.jar:7.10.2]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.10.2.jar:7.10.2]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1518) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) [netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
....
at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: org.elasticsearch.ElasticsearchSecurityException: [org.ldaptive.LdapException@1988524820::resultCode=UNWILLING_TO_PERFORM, matchedDn=null, responseControls=null, referralURLs=null, messageId=-1, message=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com', providerException=javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]; remaining name 'ou=People,dc=blabla,dc=com']
at com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend.authenticate(LDAPAuthenticationBackend.java:138) ~[opendistro_security-1.13.1.0.jar:1.13.1.0]
...
... 70 more
Caused by: javax.naming.OperationNotSupportedException: [LDAP: error code 53 - authentication required]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3332) ~[?:?]
...
... 70 more
[2021-06-05T17:24:11,080][DEBUG][c.a.o.s.a.BackendRegistry] [node-1] Cannot authenticate rest user USER (or add roles) with authdomain ldap/2 of [AuthDomain [backend=com.amazon.opendistroforelasticsearch.security.auth.internal.InternalAuthenticationBackend@6d387e9b, httpAuthenticator=com.amazon.opendistroforelasticsearch.security.http.HTTPBasicAuthenticator@645257a, order=1, challenge=true], AuthDomain [backend=com.amazon.dlic.auth.ldap.backend.LDAPAuthenticationBackend@624134f0, httpAuthenticator=com.amazon.opendistroforelasticsearch.security.http.HTTPBasicAuthenticator@63e84ebf, order=2, challenge=false]], try next
[2021-06-05T17:24:11,080][DEBUG][c.a.o.s.a.BackendRegistry] [node-1] User still not authenticated after checking 2 auth domains