Penid auth failed with 401 due to "Invalid serialized unsecured/JWS/JWE object: Missing part delimiters"

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch and Dashboard both in version 2.18.0

Describe the issue:
Opensearch backend and dashboard in version 2.18.0
configured each-node tls cert and keys, configured both basic auth and openid in dashboard and backend.
configured my IDP to send back JWT token as both ID Token and Auth Token.

After auth, my IDP always response “success” and issued JWT token.
however Opensearch always anwsers:

{
    "statusCode": 401,
    "error": "Unauthorized",
    "message": "Unauthorized"
}

and the browser address bar shows path: <myopensearch-dashboard-url>/auth/openid/login?code=5a30144b53d56622c72e5657d2bd652a&state=TSuVruLpM0bKwSuTlhaYs3.

Tried to use curl to get auth from IDP, using JWT format Auth Token to connect to opensearch backend directly, also get the same 401 response.

setup the log4j to trace level by:

logger.securityjwt.name = com.amazon.dlic.auth.http.jwt
logger.securityjwt.level = trace

then find below trace level message.
I see the AbstractHTTPJwtAuthenticator trys to parse a opaque like string as JWT, however it failed in the end.
But in my OIDC, both id_token and access_token are already issued as JWT token, so I don’t know where does this opaque like string come from.
this string is also not my basic auth admin password.

[2024-12-02T07:24:08,434][WARN ][o.o.s.h.HTTPBasicAuthenticator] [ssdl-app-logging-opensearch-manager-1] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2024-12-02T07:24:08,446][TRACE][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [ssdl-app-logging-opensearch-manager-1] Extracting JWT token from NWM2MjY1MzgtZDM1NS00ZGZhLWExYWItOGU4NjYwMTljM2Q4Tzd0bnRaazZPTHFyRFgzTGpCeG1aTHVYUGdpX0dZeldLTmhIUno0ZGdOYw failed
com.amazon.dlic.auth.http.jwt.keybyoidc.BadCredentialsException: Invalid serialized unsecured/JWS/JWE object: Missing part delimiters
	at com.amazon.dlic.auth.http.jwt.keybyoidc.JwtVerifier.getVerifiedJwtToken(JwtVerifier.java:80) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator.extractCredentials0(AbstractHTTPJwtAuthenticator.java:130) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator$1.run(AbstractHTTPJwtAuthenticator.java:111) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator$1.run(AbstractHTTPJwtAuthenticator.java:108) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) [?:?]
	at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator.extractCredentials(AbstractHTTPJwtAuthenticator.java:108) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:280) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.filter.SecurityRestFilter.checkAndAuthenticateRequest(SecurityRestFilter.java:308) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:91) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:38) [opensearch-security-2.18.0.0.jar:2.18.0.0]
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1503) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1366) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1415) [netty-handler-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.114.Final.jar:4.1.114.Final]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.text.ParseException: Invalid serialized unsecured/JWS/JWE object: Missing part delimiters
	at com.nimbusds.jose.JOSEObject.split(JOSEObject.java:226) ~[nimbus-jose-jwt-9.41.2.jar:9.41.2]
	at com.nimbusds.jwt.SignedJWT.parse(SignedJWT.java:129) ~[nimbus-jose-jwt-9.41.2.jar:9.41.2]
	at com.amazon.dlic.auth.http.jwt.keybyoidc.JwtVerifier.getVerifiedJwtToken(JwtVerifier.java:54) ~[opensearch-security-2.18.0.0.jar:2.18.0.0]
	... 47 more

Configuration:

_meta:
  type: config
  config_version: 2
config:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: 192\.168\.0\.10|192\.168\.0\.11
    authc:
      basic_internal_auth_domain:
        description: Authenticate via HTTP Basic against internal users database
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          config:
            jwt_header: Authorization
            subject_key: sub
            roles_key: groups
            openid_connect_url: https://<myidpurl>.com/.well-known/openid-configuration
            openid_connect_idp:
              enable_ssl: true
              verify_hostnames: false
          challenge: false
        authentication_backend:
          type: noop
      saml_auth_domain:
        http_enabled: false
        transport_enabled: false
      jwt_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 3
      proxy_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 4
      clientcert_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 5
      ldap:
        http_enabled: false
        transport_enabled: false
        order: 6
      kerberos_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 7
    filtered_alias_mode: disallow
    kibana:
      multitenancy_enabled: false
    authz:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: noop

Relevant Logs or Screenshots:

the issue doesn’t appear anymore if I move the openid order to 0.

@latituder Thanks for sharing your solution. However, in my case OpenID works with order 1 and 0.