# Login to kibana through JWT

**URL:** <https://forum.opensearch.org/t/login-to-kibana-through-jwt/242>\
**Category:** Security\
**Created:** [March 22, 2019, 11:43am UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242 "2019-03-22T11:43:34Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 11:43am UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/1 "2019-03-22T11:43:34Z")

</div>

Hi

The main question is:  
Is it possible to login to Kibana with JWT?  
I’m talking about functionality like described here [JWT URL Parameter Authentication with the Search Guard Kibana Plugin | Security for Elasticsearch | Search Guard](https://docs.search-guard.com/latest/kibana-authentication-jwt) (does OpenDistro have the same functionality?)

1. With the configuration below kibana can’t start because Kibana tries to auth in Elastic with basic auth but my security config allows only JWT.  
logs

```auto
odfe-node1 | [2019-03-21T18:07:15,713][WARN][c.a.d.a.h.j.HTTPJwtAuthenticator] [jDe_UcC] No Bearer scheme found in header
odfe-node1 | [2019-03-21T18:07:15,713][WARN][c.a.o.s.a.BackendRegistry] [jDe_UcC] Authentication finally failed for null from 192.168.0.2:59234

```

- How can I start kibana using only JWT
- How can I log in to kibana using JWT  
for example

```auto
http://127.0.0.1:5601?jwtToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw

```

Security config

```auto
opendistro_security:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      basic_internal_auth_domain:
        http_enabled: false
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      jwt_auth_domain:
        enabled: true
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: qwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewq
            # jwt_header: "Authorization: Bearer <token>"
            jwt_url_parameter: "jwtToken"
            roles_key: rolesKey
            subject_key: subjectKey
        authentication_backend:
          type: noop
    authz:
      roles_from_myldap:
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: noop
      roles_from_another_ldap:
        enabled: false
        authorization_backend:
          type: noop

```

1. Also I even can’t connect to elastic  
a request sample

```auto
curl -X GET \
  'https://127.0.0.1:9200' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw' \
  -H 'Postman-Token: 79519d07-ea3c-4f3e-819a-0b71964f7653' \
  -H 'cache-control: no-cache'

```

respone

```auto
odfe-node1 | [2019-03-21T17:57:44,940][WARN][c.a.o.s.a.BackendRegistry] [jDe_UcC] Authentication finally failed for null from 192.168.0.1:53128

```

---

<div class="post-metadata">

**Author:** ![elifish](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/elifish/32/1451_2.png) [@elifish](https://forum.opensearch.org/u/elifish)\
**Post date:** [March 22, 2019, 2:18pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/2 "2019-03-22T14:18:31Z")

</div>

Hi @s.samoilenko can you try to add the `--insecure` flag to your curl request. The demo SSL certificate is self-signed so curl cannot do the certificate validation. Alternatively you can use the demo root-ca file and use the `--cacert` flag in the curl request. Try the same curl request and share the output but like

```auto
curl -X GET \
  'https://127.0.0.1:9200' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw' \
  -H 'Postman-Token: 79519d07-ea3c-4f3e-819a-0b71964f7653' \
  -H 'cache-control: no-cache' --insecure 

```

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 2:39pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/3 "2019-03-22T14:39:31Z")

</div>

Oh, It’s just Postman did not convert my request properly. SSL verification was switched off in Postman.  
Anyway I run in the console

```auto
curl -X GET \
  'https://127.0.0.1:9200' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw' \
  -H 'Postman-Token: 79519d07-ea3c-4f3e-819a-0b71964f7653' \
  -H 'cache-control: no-cache' --insecure 

```

and got a response

```auto
Authentication finally failed

```

Elasticsearch container logs are

```auto
odfe-node1 | [2019-03-22T14:34:19,883][WARN][c.a.o.s.a.BackendRegistry] [RudU4k0] Authentication finally failed for null from 192.168.224.1:35308

```

---

<div class="post-metadata">

**Author:** ![elifish](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/elifish/32/1451_2.png) [@elifish](https://forum.opensearch.org/u/elifish)\
**Post date:** [March 22, 2019, 2:49pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/4 "2019-03-22T14:49:08Z")

</div>

What does your base64 encoded payload look like when you decode it? For example:

```nohighlight
{
  "subjectKey": "s.samoilenko",
  "rolesKey": "admin"
}

```

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 3:06pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/5 "2019-03-22T15:06:53Z")

</div>

yep, the wrong token.  
Tried to do the same request with new one

```auto
curl -X GET 'https://127.0.0.1:9200' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw' -H 'Postman-Token: 79519d07-ea3c-4f3e-819a-0b71964f7653' -H 'cache-control: no-cache' --insecure

```

and got the same error

The token contains

```auto
{
  "rolesKey": "all_access",
  "subjectKey": "admin",
}

```

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 3:13pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/6 "2019-03-22T15:13:03Z")

</div>

my docker-compose.yml

```auto
version: '3'
services:
  odfe-node1:
    image: amazon/opendistro-for-elasticsearch:0.7.0
    container_name: odfe-node1
    environment:
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./config/root-ca.pem:/usr/share/elasticsearch/config/root-ca.pem
      - ./config/esnode.pem:/usr/share/elasticsearch/config/esnode.pem
      - ./config/esnode-key.pem:/usr/share/elasticsearch/config/esnode-key.pem
      - ./config/kirk.pem:/usr/share/elasticsearch/config/kirk.pem
      - ./config/kirk-key.pem:/usr/share/elasticsearch/config/kirk-key.pem
      - odfe-data1:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./security.config.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - odfe-net
    ports:
      - 9200:9200

  odfe-node2:
    image: amazon/opendistro-for-elasticsearch:0.7.0
    container_name: odfe-node2
    environment:
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.zen.ping.unicast.hosts=odfe-node1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./config/root-ca.pem:/usr/share/elasticsearch/config/root-ca.pem
      - ./config/esnode.pem:/usr/share/elasticsearch/config/esnode.pem
      - ./config/esnode-key.pem:/usr/share/elasticsearch/config/esnode-key.pem
      - ./config/kirk.pem:/usr/share/elasticsearch/config/kirk.pem
      - ./config/kirk-key.pem:/usr/share/elasticsearch/config/kirk-key.pem
      - odfe-data2:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./security.config.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml
    networks:
      - odfe-net
  kibana:
    image: amazon/opendistro-for-elasticsearch-kibana:0.7.0
    container_name: odfe-kibana
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      ELASTICSEARCH_URL: https://odfe-node1:9200
    volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml
    networks:
      - odfe-net

volumes:
  odfe-data1:
  odfe-data2:

networks:
  odfe-net:

```

---

<div class="post-metadata">

**Author:** ![elifish](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/elifish/32/1451_2.png) [@elifish](https://forum.opensearch.org/u/elifish)\
**Post date:** [March 22, 2019, 3:17pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/7 "2019-03-22T15:17:01Z")

</div>

Can you `null` out the jwt url parameter and set the header to just `Authorization`? It looks like you are passing the token in the header. See below

```auto
            jwt_header: "Authorization"
            jwt_url_parameter: null
            roles_key: rolesKey
            subject_key: subjectKey

```

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 3:33pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/8 "2019-03-22T15:33:06Z")

</div>

I’ve changed my security.config.yml to

```auto
      jwt_auth_domain:
        enabled: true
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: qwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewq
            jwt_header: "Authorization"
            jwt_url_parameter: null
            roles_key: rolesKey
            subject_key: subjectKey

```

then run

```auto
docker-compose down -v
docker-compose up --build

curl -X GET 'https://127.0.0.1:9200' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6IjU2ZTE3OTE4LTA2Y2UtYTJlMS1kY2RmLTgyN2M3YjAzNjU4OCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzE4OTQ2MiwiZXhwIjoxNTUzMzYyMjYyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.mU9XEYq0B0cQTIvNNND1M_tsTS35NeZAL5suCoQbunw' -H 'Postman-Token: 79519d07-ea3c-4f3e-819a-0b71964f7653' -H 'cache-control: no-cache' --insecure

```

and got the same error

```auto
odfe-node1 | [2019-03-22T15:27:16,576][WARN][c.a.o.s.a.BackendRegistry] [CR2zNTJ] Authentication finally failed for null from 172.24.0.1:46042

```

I even tried to remove `Bearer` from the header.  
and it looks like opendistro see the header

```auto
odfe-node1 | [2019-03-22T15:29:47,899][WARN][c.a.d.a.h.j.HTTPJwtAuthenticator] [CR2zNTJ] No Bearer scheme found in header
odfe-node1 | [2019-03-22T15:29:47,900][WARN][c.a.o.s.a.BackendRegistry] [CR2zNTJ] Authentication finally failed for null from 172.24.0.1:46650

```

---

<div class="post-metadata">

**Author:** ![elifish](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/elifish/32/1451_2.png) [@elifish](https://forum.opensearch.org/u/elifish)\
**Post date:** [March 22, 2019, 4:08pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/9 "2019-03-22T16:08:15Z")

</div>

Definitely keep the `Bearer` portion of the header your first CURL looks right. Do you have the issuer field and expiration time in the payload? Those would be

```auto
{
  "rolesKey": "all_access",
  "subjectKey": "admin",
  "iss": "example.com",
  "exp": 1554891380,
}

```

1. How are you signing your token?
2. Can you provide your elasticsearch and kibana.ymls you are using?

Thanks

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 4:32pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/10 "2019-03-22T16:32:46Z")

</div>

I use the jsonwebtoken lib (nodejs) to generate JWT token

issuer `http://localhost`  
expiration `2days`

secret `qwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewqqwertyuiopasdfghjklzxcvbnmnbvcxzasdfghjklpoiuytrewq`

I’ve tried to remove the issuer but got the same error.

```auto
jwt.sign(
			{
				id,
				accountId,
				token,
				rolesKey: 'all_access',
				subjectKey: 'admin'
			},
			secret,
			{
				expiresIn,
				issuer
			}
		);

```

raw token

```auto
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6ImNhY2IzYzFlLTQ4NWUtNzZjYS03Njg1LTNiZmQ5ZDA2ZDAxOCIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzI3MTMxMiwiZXhwIjoxNTUzNDQ0MTEyLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.lcZvBrgWx1eFe1fuxlgE3tC0hXUZE39TO-snYwyw0PM

```

decoded token header and payload

```auto
{
  "alg": "HS256",
  "typ": "JWT"
}
{
  "id": "be2772ce10184cfcafdae99cde7944e7",
  "accountId": "be2772ce10184cfcafdae99cde7944e7",
  "token": "cacb3c1e-485e-76ca-7685-3bfd9d06d018",
  "rolesKey": "all_access",
  "subjectKey": "admin",
  "iat": 1553271312,
  "exp": 1553444112,
  "iss": "http://localhost"
}

```

elasticsearch.yml

```auto
cluster.name: "docker-cluster"
network.host: 0.0.0.0

# minimum_master_nodes need to be explicitly set when bound on a public IP
# set to 1 to allow single node clusters
# Details: https://github.com/elastic/elasticsearch/pull/17288
discovery.zen.minimum_master_nodes: 1

######## Start OpenDistro for Elasticsearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
opendistro_security.ssl.transport.pemcert_filepath: esnode.pem
opendistro_security.ssl.transport.pemkey_filepath: esnode-key.pem
opendistro_security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
opendistro_security.ssl.transport.enforce_hostname_verification: false
opendistro_security.ssl.http.enabled: true
opendistro_security.ssl.http.pemcert_filepath: esnode.pem
opendistro_security.ssl.http.pemkey_filepath: esnode-key.pem
opendistro_security.ssl.http.pemtrustedcas_filepath: root-ca.pem
opendistro_security.allow_unsafe_democertificates: true
opendistro_security.allow_default_init_securityindex: true
opendistro_security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

opendistro_security.audit.type: internal_elasticsearch
opendistro_security.enable_snapshot_restore_privilege: true
opendistro_security.check_snapshot_restore_write_privileges: true
opendistro_security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
opendistro_security.cache.ttl_minutes: 0
cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3
######## End OpenDistro for Elasticsearch Security Demo Configuration ########

```

kibana.yml

```auto
# Default Kibana configuration from kibana-docker.

server.name: kibana
server.host: "0"
elasticsearch.url: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization","jwtToken"]

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]
opendistro_security.jwt.enabled: true

```

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 22, 2019, 5:04pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/11 "2019-03-22T17:04:33Z")

</div>

If I found a right place then it can’t get credentils

file `src/main/java/com/amazon/opendistroforelasticsearch/security/auth/BackendRegistry.java`

`authCredenetials` is null

```auto
            log.warn("Authentication finally failed for {} from {}", authCredenetials == null ? null:authCredenetials.getUsername(), remoteAddress);
            auditLog.logFailedLogin(authCredenetials == null ? null:authCredenetials.getUsername(), false, null, request);
            channel.sendResponse(new BytesRestResponse(RestStatus.UNAUTHORIZED, "Authentication finally failed"));
            return false;

```

---

<div class="post-metadata">

**Author:** ![nprashar](https://avatars.discourse-cdn.com/v4/letter/n/a3d4f5/32.png) [@nprashar](https://forum.opensearch.org/u/nprashar)\
**Post date:** [March 26, 2019, 4:30pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/12 "2019-03-26T16:30:27Z")

</div>

Hi there,

I’m trying to get you steps for kibana, but maybe we can first check if you’re able to connect using jwt to ES:

The following configuration works for me:

**I generated my JWT Token using jjwt library([GitHub - jwtk/jjwt: Java JWT: JSON Web Token for Java and Android](https://github.com/jwtk/jjwt#jws-read))**

You will need to decode your private key with base64 and put it in the signing\_key value below.

**This is the sample code for me for generating a JWT token - in java - notice I’m setting roles as admin, and subject as admin**

```
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
import io.jsonwebtoken.security.Keys;
import java.security.Key;
import java.util.Date;
import java.util.HashMap;
import io.jsonwebtoken.io.Encoders;

public class JWTTest {
public static void main(String[] args) {
Key key = Keys.secretKeyFor(SignatureAlgorithm.HS256);
        Date exp = new Date(System.currentTimeMillis() + 1000000);
        HashMap<String,Object> hm = new HashMap<>();
        hm.put("roles","admin");
        String jws = Jwts.builder()
                .setClaims(hm)
                .setIssuer("https://localhost")
                .setSubject("admin")
                .setExpiration(exp)
                .signWith(key).compact();
        System.out.println(jws);
        String encoded = Encoders.BASE64.encode(key.getEncoded());
       // Need to put this in the signing_key
        System.out.println(encoded);
    }
}

```

**securityconfig/config.yml - notice the roles\_key set to roles, and subject\_key set to sub**

```
 jwt_auth_domain:
    enabled: true
    http_enabled: true
    transport_enabled: true
    order: 0
    http_authenticator:
      type: jwt
      challenge: false
      config:
        signing_key: "IXIENkVkTX6+QS1NVntGWIvYa7h8JC5ONZpegpkuUw0="
        jwt_header: "Authorization"
        roles_key: "roles"
        subject_key: "sub"
    authentication_backend:
      type: noop

```

**Make sure you reinitialize the index in the docker shell**

`plugins/opendistro_security/tools/securityadmin.sh -f plugins/opendistro_security/securityconfig/config.yml -icl -nhnv -cert config/kirk.pem -cacert config/root-ca.pem -key config/kirk-key.pem -t config`

**The following is how I’m calling the service**

`curl -XGET https://localhost:9200/_cat/nodes -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJyb2xlcyI6ImFkbWluIiwiaXNzIjoiaHR0cHM6Ly9sb2NhbGhvc3QiLCJzdWIiOiJhZG1pbiIsImV4cCI6MTU1MzYxNzcwNH0.HMFmwkNEkJkOaK_0ALAKbN9aPO0SYwxVtS1Z5YsiTKI" --insecure`  
**172.17.0.2 21 45 3 0.07 0.06 0.09 mdi \* vDaYZ9h**

Can you try the above and see if this works ?

---

<div class="post-metadata">

**Author:** ![nprashar](https://avatars.discourse-cdn.com/v4/letter/n/a3d4f5/32.png) [@nprashar](https://forum.opensearch.org/u/nprashar)\
**Post date:** [March 26, 2019, 8:53pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/13 "2019-03-26T20:53:16Z")

</div>

I’m also able to run Kibana now with one additional line change in my kibana.yml …

```
# Default Kibana configuration from kibana-docker.

server.name: kibana
server.host: "0"
elasticsearch.url: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]

**opendistro_security.auth.type: "jwt"**

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]

```

I’m quering kibana like the following:

```
curl -XGET http://localhost:5601 -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJyb2xlcyI6ImFkbWluIiwiaXNzIjoiaHR0cHM6Ly9sb2NhbGhvc3QiLCJzdWIiOiJhZG1pbiIsImV4cCI6MTU1MzY0Mjc1NX0.2RVy0VEObwduF9nNZas498LTJMRLC9luTuebMOyhD-g" -i
HTTP/1.1 302 Found
location: /app/kibana
kbn-name: kibana
set-cookie: security_storage=Fe26.2 **a86a495463a9ed2aef99e9499025b000888bc70232d006765c9990f8c9d7412*viOmkphhLLIDeBTxX9_OkQ*lIBpboN6gQ07QvwY7mMp-48IsrvI0qtfaRR8_VmPesYmlqlNizId2smn-kXtIJdsmZBpz7y4WLJzmqP0hKKCBAAJ9Bccj-fVh5QJdHW6mWEhuS870VlB9PUMZAnQ8ju6D8Gs-70A16rodBDSI4b601EhJET4vtMObTFmvYkiavqKvc9CPbwMpHRQdIKwX9AzSjbekMC8CSn1PgzMbtNijYNFd3sLZHrDxrqTSQijm8M** ba624f98f91081024b49264a08c692287b30bca4f185aa8925c1bb238cdf27ef*fc9z6yinUj2Xp920Iy-GoKdVzO5G4aZRsxQWi_bVH-Y; Path=/
set-cookie: security_preferences=Fe26.2 **a2791807692cd418aa644804fd0e6e5cd33421a899e0797d8a97ec4e7f2cbf0*guZ5n6zMcCwylCPOazyyew*1n43XcDV1NcGvgl-VwD07njHLkxn-VdgQNVMk5ZQSsw** f25a10407839cc2869b06826eb5459f166baf6fcea11df6b1f4a316152fec3e4*K5wr95D7cVoetpvEFjdzjSN-mgvBEU9tWpx6QiLgEuE; Max-Age=2217100485; Expires=Mon, 27 Jun 2089 17:56:50 GMT; Path=/
set-cookie: security_authentication=Fe26.2 **5ca6f12884a00a406f89887bb91f33ee7a68f22c815996a9adbda934698364d*OuII1jATnWfYzaHIv4_HvQ*qoTlwVqRvpDzkWmq-JYZbXpSbEJ6DyG5qhmNenM0GB6vbGEcnkXmpUFvOICkAyRuzmKwl9Uut1GYM98TLwhTZbzFb6Z1d5Sb4MOpk6DJNFjuokIm0u9tqsCwCGMEO_avmosVy4gceAluSX-7vN-vC461jt2B3_DIbyeREjPLtjr91a2I95nGQRir_-4cypkjUaS3Blub1ZC7fNnkBcK5POvo-nKTXJmx5KQx4O_6zVc3vFfoQLJ7_AUrLAID_htMHMv5o7_qn1oMHP-LTr5zvO4iDLlY1UgBJCmikpMatxPg8ophKxWkMRuIdo4UaZEjrzXwQPJtYBmpJxwQtolJQB5jwOnNNVqtUeiI7sWitHM** 1c4cf336b71a513045bf0bfe50ff96447c213f70dfd3745d713e57235a7edff9*fLp9DLSMhgKHjOIJ8VDHMbVI9Z7W56Velx4Pi5STK4s; Max-Age=3600; Expires=Tue, 26 Mar 2019 21:42:05 GMT; HttpOnly; Path=/
cache-control: no-cache
content-length: 0
connection: close
Date: Tue, 26 Mar 2019 20:42:05 GMT

```

If you want to make additional calls, to kibana, you’ll need to set the security\_storage, security\_preferences, and security\_authentication to any follow-up requests. I was using manually setting these to see it in action.

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 27, 2019, 9:50am UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/14 "2019-03-27T09:50:53Z")

</div>

Thank you.  
I forgot to encode my secret wtih base64.

---

<div class="post-metadata">

**Author:** ![s.samoilenko](https://avatars.discourse-cdn.com/v4/letter/s/47e85d/32.png) [@s.samoilenko](https://forum.opensearch.org/u/s.samoilenko)\
**Post date:** [March 27, 2019, 9:53am UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/15 "2019-03-27T09:53:39Z")

</div>

I expanded my kibana.yml with following parameters

```auto
opendistro_security.auth.type: "jwt"
opendistro_security.jwt.url_param: jwtToken

```

and now I can auth to kibana like

```auto
http://127.0.0.1:5601?jwtToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJlMjc3MmNlMTAxODRjZmNhZmRhZTk5Y2RlNzk0NGU3IiwiYWNjb3VudElkIjoiYmUyNzcyY2UxMDE4NGNmY2FmZGFlOTljZGU3OTQ0ZTciLCJ0b2tlbiI6ImE0MWUxNjQ1LWQxOTktNDFjZS1hZjAxLTgxZmUyZjQ3MjczYiIsInJvbGVzS2V5IjoiYWxsX2FjY2VzcyIsInN1YmplY3RLZXkiOiJhZG1pbiIsImlhdCI6MTU1MzY3NzI5NSwiZXhwIjoxNTUzODUwMDk1LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0In0.sbtKb66PN3Qxt9njcLG56_n4m6VkTiGFHAwrTEJd64k

```

---

<div class="post-metadata">

**Author:** ![darkice01](https://avatars.discourse-cdn.com/v4/letter/d/a587f6/32.png) [@darkice01](https://forum.opensearch.org/u/darkice01)\
**Post date:** [February 10, 2020, 4:28pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/16 "2020-02-10T16:28:48Z")

</div>

I’m also experiencing this problem. I tried to follow these examples, but nothing worked.

I tried following this example, it does not work for me.

> **[Use JSON Web Tokens (JWTs) to Authenticate in Open Distro for Elasticsearch...](https://aws.amazon.com/it/blogs/opensource/json-web-tokens-jwt-authenticate-open-distro-for-elasticsearch-kibana/)**
>
> 中文版 – Token-based authentication systems are popular in the world of web services. They provide many benefits, including (but not limited to) security, scalability, statelessness, and extensibility. With Amazon’s Open Distro for Elasticsearch, users...

I’m using this

**docker-compose.xml**

```
version: '3'
services:
  odfe-node1:
    image: amazon/opendistro-for-elasticsearch:1.3.0
    container_name: odfe-node1
    environment:
      - cluster.name=odfe-cluster
      - node.name=odfe-node1
      - discovery.type=single-node
	#- discovery.seed_hosts=odfe-node1,odfe-node2
	#- cluster.initial_master_nodes=odfe-node1,odfe-node2
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    ulimits:
      memlock:
	soft: -1
	hard: -1
      nofile:
	soft: 65536 # maximum number of open files for the Elasticsearch user, set to at least 65536 on modern systems
	hard: 65536
    volumes:
      - /home/davidemalagoli/kibana/custom-docker/community/config.yml:/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml
      - odfe-data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - odfe-net
	
  kibana:
    image: amazon/opendistro-for-elasticsearch-kibana:1.3.0
    container_name: odfe-kibana
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      ELASTICSEARCH_URL: https://odfe-node1:9200
      ELASTICSEARCH_HOSTS: https://odfe-node1:9200
    networks:
      - odfe-net

volumes:
  odfe-data1:
  odfe-data2:

networks:
  odfe-net:

```

and following

**config.xml**

```
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
	enabled: false
    authc:
      basic_internal_auth_domain:
	description: "Authenticate via HTTP Basic against internal users database"
	http_enabled: false
	transport_enabled: false
	order: 4
	http_authenticator:
	  type: basic
	  challenge: true
	authentication_backend:
	  type: intern
      jwt_auth_domain:
	http_enabled: true
	transport_enabled: false
	order: 0
	http_authenticator:
	  type: jwt
	  challenge: false
	  config:
	    signing_key: "c2VjcmV0"
	    jwt_header: null
	    jwt_url_parameter: "jwt"
	    roles_key: "roles"
	    subject_key: "sub"
	authentication_backend:
	  type: noop
    authz:
      roles_from_myldap:
	description: "Authorize via LDAP or Active Directory"
	http_enabled: false
	transport_enabled: false
	authorization_backend:
	  # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
	  type: ldap
	  config:
	    # enable ldaps
	    enable_ssl: false
	    # enable start tls, enable_ssl should be false
	    enable_start_tls: false
	    # send client certificate
	    enable_ssl_client_auth: false
	    # verify ldap hostname
	    verify_hostnames: true
	    hosts:
	    - localhost:8389
	    bind_dn: null
	    password: null
	    rolebase: 'ou=groups,dc=example,dc=com'
	    # Filter to search for roles (currently in the whole subtree beneath rolebase)
	    # {0} is substituted with the DN of the user
	    # {1} is substituted with the username
	    # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
	    rolesearch: '(member={0})'
	    # Specify the name of the attribute which value should be substituted with {2} above
	    userroleattribute: null
	    # Roles as an attribute of the user entry
	    userrolename: disabled
	    #userrolename: memberOf
	    # The attribute in a role entry containing the name of that role, Default is "name".
	    # Can also be "dn" to use the full DN as rolename.
	    rolename: cn
	    # Resolve nested roles transitive (roles which are members of other roles and so on ...)
	    resolve_nested_roles: true
	    userbase: 'ou=people,dc=example,dc=com'
	    # Filter to search for users (currently in the whole subtree beneath userbase)
	    # {0} is substituted with the username
	    usersearch: '(uid={0})'
	    # Skip users matching a user name, a wildcard or a regex pattern
	    #skip_users:
	    # - 'cn=Michael Jackson,ou*people,o=TEST'
	    # - '/\S*/'
      roles_from_another_ldap:
	description: "Authorize via another Active Directory"
	http_enabled: false
	transport_enabled: false
	authorization_backend:
	  type: ldap
	  #config goes here ...
  # auth_failure_listeners:
  # ip_rate_limiting:
  # type: ip
  # allowed_tries: 10
  # time_window_seconds: 3600
  # block_expiry_seconds: 600
  # max_blocked_clients: 100000
  # max_tracked_clients: 100000
  # internal_authentication_backend_limiting:
  # type: username
  # authentication_backend: intern
  # allowed_tries: 10
  # time_window_seconds: 3600
  # block_expiry_seconds: 600
  # max_blocked_clients: 100000
  # max_tracked_clients: 100000

```

In no way I am able to obtain a positive response from

`curl -v -XGET https://localhost:9200/_cat/nodes?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsIm5hbWUiOiJKb2huIERvZSIsImlhdCI6MTUxNjIzOTAyMiwicm9sZXMiOiJhZG1pbiJ9.SPgn7KcBxJRHzUqO-65mElicejY8zxo-Znn2cNtFAjs --insecure`

I validated the JWT contents using

> **[JWT.IO](https://jwt.io/)**
>
> JSON Web Tokens are an open, industry standard RFC 7519 method for representing claims securely between two parties.

```
{
  "sub": "admin",
 "name": "John Doe",
 "iat": 1516239022,
"roles": "admin"
}

```

Can anyone help me, please?

Thank you

---

<div class="post-metadata">

**Author:** ![yashreddy](https://avatars.discourse-cdn.com/v4/letter/y/a9adbd/32.png) [@yashreddy](https://forum.opensearch.org/u/yashreddy)\
**Post date:** [March 11, 2020, 7:54am UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/17 "2020-03-11T07:54:45Z")

</div>

Facing this issue, Can anybody help me  
[https://stackoverflow.com/questions/60622018/opendistro-throwing-an-error-on-accepting-a-jwt-token-from-keycloak](https://stackoverflow.com/questions/60622018/opendistro-throwing-an-error-on-accepting-a-jwt-token-from-keycloak)

---

<div class="post-metadata">

**Author:** ![yashreddy](https://avatars.discourse-cdn.com/v4/letter/y/a9adbd/32.png) [@yashreddy](https://forum.opensearch.org/u/yashreddy)\
**Post date:** [March 11, 2020, 5:56pm UTC](https://forum.opensearch.org/t/login-to-kibana-through-jwt/242/18 "2020-03-11T17:56:33Z")

</div>

Please find the clear question  
I am buidling a setup where there are three components:

1. Keycloak as An IDP using openid connect , as well as client authentication method as client\_secret.
2. Opendistro security plugin with JWT token configuration.
3. Python code trying to make an API call from access token got through curl command.

OpenDistro security plugin configuration:

```auto
---

---

# This is the main Open Distro Security configuration file where authentication
# and authorization is defined.
#
# You need to configure at least one authentication domain in the authc of this file.
# An authentication domain is responsible for extracting the user credentials from
# the request and for validating them against an authentication backend like Active Directory for example.
#
# If more than one authentication domain is configured the first one which succeeds wins.
# If all authentication domains fail then the request is unauthenticated.
# In this case an exception is thrown and/or the HTTP status is set to 401.
#
# After authentication authorization (authz) will be applied. There can be zero or more authorizers which collect
# the roles from a given backend for the authenticated user.
#
# Both, authc and auth can be enabled/disabled separately for REST and TRANSPORT layer. Default is true for both.
# http_enabled: true
# transport_enabled: true
#
# For HTTP it is possible to allow anonymous authentication. If that is the case then the HTTP authenticators try to
# find user credentials in the HTTP request. If credentials are found then the user gets regularly authenticated.
# If none can be found the user will be authenticated as an "anonymous" user. This user has always the username "anonymous"
# and one role named "anonymous_backendrole".
# If you enable anonymous authentication all HTTP authenticators will not challenge.
#
#
# Note: If you define more than one HTTP authenticators make sure to put non-challenging authenticators like "proxy" or "clientcert"
# first and the challenging one last.
# Because it's not possible to challenge a client with two different authentication methods (for example
# Kerberos and Basic) only one can have the challenge flag set to true. You can cope with this situation
# by using pre-authentication, e.g. sending a HTTP Basic authentication header in the request.
#
# Default value of the challenge flag is true.
#
#
# HTTP
# basic (challenging)
# proxy (not challenging, needs xff)
# kerberos (challenging)
# clientcert (not challenging, needs https)
# jwt (not challenging)
# host (not challenging) #DEPRECATED, will be removed in a future version.
# host based authentication is configurable in roles_mapping

# Authc
# internal
# noop
# ldap

# Authz
# ldap
# noop

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    # Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
    #filtered_alias_mode: warn
    #do_not_fail_on_forbidden: false
    #kibana:
    # Kibana multitenancy
    #multitenancy_enabled: true
    #server_username: kibanaserver
    #index: '.kibana'
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        #remoteIpHeader: 'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
      kerberos_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 6
        http_authenticator:
          type: kerberos
          challenge: true
          config:
            # If true a lot of kerberos/security related debugging output will be logged to standard out
            krb_debug: false
            # If true then the realm will be stripped from the user name
            strip_realm_from_principal: true
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      proxy_auth_domain:
        description: "Authenticate via proxy"
        http_enabled: false
        transport_enabled: false
        order: 3
        http_authenticator:
          type: proxy
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: noop
      jwt_auth_domain:
        description: "Authenticate via Json Web Token"
        http_enabled: true
        transport_enabled: false
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: "-----BEGIN PUBLIC KEY-----MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAt6GjgJTJUztz7ATm3+vYPGnnotQ5hFX57OL1UkLoqVKr+sfS6DxCl9f4/2L818TLYbtn0h9GnUOf/8XJLQpWy+B6uK3X0vVMJyzYqIaRrhUStZTXUnxOwSCzEk2sq7MT2akStA4QwtsJ9b6YbkWq4Tp0ml0kWYsqIDELnza7YJRjmDR/AFPbqiFdCANFpUtuq/heiVux9vZb881dYUmkcYtC3voH2J+KYAoCmljOtg33ioeVBLkrhzGW/7u2XJ+ELTrk7ZrXrDxpQmYaL3aojMv60BFI+WoY4xFjrTai2D1fao32I6sOcITyY6K74HCo+Fyx6cc6UiJEO5bDGrfFNQIDAQAB-----END PUBLIC KEY-----"
            jwt_header: "Authorization"
            jwt_url_parameter: null
            roles_key: roles
            subject_key: preferred_username
            openid_connect_url: "http://10.95.126.45:8080/auth/realms/demo/protocol/openid-connect/certs"
        authentication_backend:
          type: noop
      clientcert_auth_domain:
        description: "Authenticate via SSL client certificates"
        http_enabled: false
        transport_enabled: false
        order: 2
        http_authenticator:
          type: clientcert
          config:
            username_attribute: cn #optional, if omitted DN becomes username
          challenge: false
        authentication_backend:
          type: noop
      saml_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 8
        http_authenticator:
          type: saml
          challenge: true
          config:
            idp:
              metadata_url: http://10.95.126.45:8080/auth/realms/demo/protocol/saml/descriptor
              entity_id: http://10.95.126.45:8080/auth/realms/demo
            sp:
              entity_id: kibana-saml
            #kibana_url: http://10.95.126.45:5601
            roles_key: Role
            exchange_key: E3B0C44298FC000000000000000000000000000000000000A495991B7852B855
        authentication_backend:
            type: noop
      ldap:
        description: "Authenticate via LDAP or Active Directory"
        http_enabled: false
        transport_enabled: false
        order: 5
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          # LDAP authentication backend (authenticate users against a LDAP or Active Directory)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - localhost:8389
            bind_dn: null
            password: null
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(sAMAccountName={0})'
            # Use this attribute from the user as username (if not set then DN is used)
            username_attribute: null
    authz:
      roles_from_myldap:
        description: "Authorize via LDAP or Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
            - localhost:8389
            bind_dn: null
            password: null
            rolebase: 'ou=groups,dc=example,dc=com'
            # Filter to search for roles (currently in the whole subtree beneath rolebase)
            # {0} is substituted with the DN of the user
            # {1} is substituted with the username
            # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
            rolesearch: '(member={0})'
            # Specify the name of the attribute which value should be substituted with {2} above
            userroleattribute: null
            # Roles as an attribute of the user entry
            userrolename: disabled
            #userrolename: memberOf
            # The attribute in a role entry containing the name of that role, Default is "name".
            # Can also be "dn" to use the full DN as rolename.
            rolename: cn
            # Resolve nested roles transitive (roles which are members of other roles and so on ...)
            resolve_nested_roles: true
            userbase: 'ou=people,dc=example,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(uid={0})'
            # Skip users matching a user name, a wildcard or a regex pattern
            #skip_users:
            # - 'cn=Michael Jackson,ou*people,o=TEST'
            # - '/\S*/'
      roles_from_another_ldap:
        description: "Authorize via another Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: ldap
          #config goes here ...
  # auth_failure_listeners:
  # ip_rate_limiting:
  # type: ip
  # allowed_tries: 10
  # time_window_seconds: 3600
  # block_expiry_seconds: 600
  # max_blocked_clients: 100000
  # max_tracked_clients: 100000
  # internal_authentication_backend_limiting:
  # type: username
  # authentication_backend: intern
  # allowed_tries: 10
  # time_window_seconds: 3600
  # block_expiry_seconds: 600
  # max_blocked_clients: 100000
  # max_tracked_clients: 100000

```

KeyCloak configuration: [KeyCloak-Client-OpenIDconnect-Client-Authentication-picture](https://i.stack.imgur.com/JgBBp.png)

[KeyCloak-Client-Configuration-page-main](https://i.stack.imgur.com/xlT3A.png)

[KeyCloak-Realm-configuration-page](https://i.stack.imgur.com/5vRI2.png)

KeyCloak OIDC JSON for reference:

```auto
{
  "realm": "demo",
  "auth-server-url": "http://10.95.126.45:8080/auth/",
  "ssl-required": "none",
  "resource": "elastic-2",
  "verify-token-audience": true,
  "credentials": {
    "secret": "9e30fff3-ab4b-4b5d-89c9-0d94cf807483"
  },
  "confidential-port": 0,
  "policy-enforcer": {}
}

```

The application i am making query is as follows:

```auto
curl http://10.95.126.45:8080/auth/realms/demo/protocol/openid-connect/token \
-H 'Cache-Control: nocache' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Postman-Token: c43306f5-f2b9-46c9-b0fb-efd63314df4c' \
-H 'cache-control: no-cache' \
-d 'grant_type=client_credentials&client_id=elastic-2&client_secret=9e30fff3-ab4b-4b5d-89c9-0d94cf807483'

```

I am able to get the access token from this command. With [jwt.io](http://jwt.io/) , verify the signature by putting the public key in keys section of realm in keycloak.

[Verification of JWT Token using the same signing key in JWT configuration of opendistro](https://i.stack.imgur.com/4JdOL.png)

Then I am quering elastic search using a python requests library.

```auto
import requests
access_token="eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJGLUpqN0VRWjU2anJoNDFrQjBsdzBoYmpreUQwNEpNVEtENDkxa0RaLTlVIn0.eyJqdGkiOiJkNzNiNWI4Ni1lNjgwLTQ0ZDMtYTgyNC00Mjk3NjVhZTM1YzkiLCJleHAiOjE1ODM4NTYwNTIsIm5iZiI6MCwiaWF0IjoxNTgzODU1NzUyLCJpc3MiOiJodHRwOi8vMTAuOTUuMTI2LjQ1OjgwODAvYXV0aC9yZWFsbXMvZGVtbyIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiJlOWI4NGJkZC1lZDgyLTQ3YWYtOTg2OC1lYzc5NDE3Y2NkOWEiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJlbGFzdGljLTIiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiI3MDcyZGRhZi1iMWVjLTQxYTUtODAwZi1lNzNlNTExMmNkMjEiLCJhY3IiOiIxIiwiYWxsb3dlZC1vcmlnaW5zIjpbIioiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJlbGFzdGljLTIiOnsicm9sZXMiOlsidW1hX3Byb3RlY3Rpb24iXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoicHJvZmlsZSBlbWFpbCIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwiY2xpZW50SG9zdCI6IjEwLjk1LjEyNi40NSIsImNsaWVudElkIjoiZWxhc3RpYy0yIiwicHJlZmVycmVkX3VzZXJuYW1lIjoic2VydmljZS1hY2NvdW50LWVsYXN0aWMtMiIsImNsaWVudEFkZHJlc3MiOiIxMC45NS4xMjYuNDUifQ.AI7_jkNAK_LD5jsgOn-wn4xdk4ZKXqAo1G7m0MjaIjco37dfxGEL-cKhDnZYvsRiULmXtsBXjBBZchOpsoyWbAczH-cNxeQ9bPwbi8saV-9sHLccTpHdZZAY_vG3JyrCaHhno4F9VnU-smks4E-fe8H-Wm1v7yfX6WiCb3txUOwE5z4VTjgNyCOsEpAVQZ2luMiwXK6a450F3DCmzg3gwfseAIPOJxfQBdGje20wK8eRMBSapEHOY75JTFM9ADjN6bOEFHhiWFeaAz0aii20tgNUcfJHCGnDPjrPdUkjy0vqYTSW1SDyhltFgoYJMF9-XWKBM3mL4ctEZw9VmKdBwA"
headers = {'Authorization': 'Bearer %s' % (access_token)}
            # YOLO
response = requests.get('https://localhost:9200/demo_index*/_search', headers=headers, verify=False)
print(response.text)

```

NOTE: The access token above is valid one you can check with Signing key of jwt opendistro and this token to see the signatures verified.

However , when run the above python script i get an error.

The error on elastic search is as follows:

```auto
[2020-03-10T11:59:06,295][INFO][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [localhost.localdomain] Extracting JWT token from eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJGLUpqN0VRWjU2anJoNDFrQjBsdzBoYmpreUQwNEpNVEtENDkxa0RaLTlVIn0.eyJqdGkiOiJkNzNiNWI4Ni1lNjgwLTQ0ZDMtYTgyNC00Mjk3NjVhZTM1YzkiLCJleHAiOjE1ODM4NTYwNTIsIm5iZiI6MCwiaWF0IjoxNTgzODU1NzUyLCJpc3MiOiJodHRwOi8vMTAuOTUuMTI2LjQ1OjgwODAvYXV0aC9yZWFsbXMvZGVtbyIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiJlOWI4NGJkZC1lZDgyLTQ3YWYtOTg2OC1lYzc5NDE3Y2NkOWEiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJlbGFzdGljLTIiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiI3MDcyZGRhZi1iMWVjLTQxYTUtODAwZi1lNzNlNTExMmNkMjEiLCJhY3IiOiIxIiwiYWxsb3dlZC1vcmlnaW5zIjpbIioiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJlbGFzdGljLTIiOnsicm9sZXMiOlsidW1hX3Byb3RlY3Rpb24iXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoicHJvZmlsZSBlbWFpbCIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwiY2xpZW50SG9zdCI6IjEwLjk1LjEyNi40NSIsImNsaWVudElkIjoiZWxhc3RpYy0yIiwicHJlZmVycmVkX3VzZXJuYW1lIjoic2VydmljZS1hY2NvdW50LWVsYXN0aWMtMiIsImNsaWVudEFkZHJlc3MiOiIxMC45NS4xMjYuNDUifQ.AI7_jkNAK_LD5jsgOn-wn4xdk4ZKXqAo1G7m0MjaIjco37dfxGEL-cKhDnZYvsRiULmXtsBXjBBZchOpsoyWbAczH-cNxeQ9bPwbi8saV-9sHLccTpHdZZAY_vG3JyrCaHhno4F9VnU-smks4E-fe8H-Wm1v7yfX6WiCb3txUOwE5z4VTjgNyCOsEpAVQZ2luMiwXK6a450F3DCmzg3gwfseAIPOJxfQBdGje20wK8eRMBSapEHOY75JTFM9ADjN6bOEFHhiWFeaAz0aii20tgNUcfJHCGnDPjrPdUkjy0vqYTSW1SDyhltFgoYJMF9-XWKBM3mL4ctEZw9VmKdBwA failed
com.amazon.dlic.auth.http.jwt.keybyoidc.BadCredentialsException: Algorithm of JWT does not match algorithm of JWK (HS512 != RS256)
        at com.amazon.dlic.auth.http.jwt.keybyoidc.JwtVerifier.validateSignatureAlgorithm(JwtVerifier.java:98) ~[opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.keybyoidc.JwtVerifier.getInitializedSignatureVerifier(JwtVerifier.java:106) ~[opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.keybyoidc.JwtVerifier.getVerifiedJwtToken(JwtVerifier.java:66) ~[opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator.extractCredentials0(AbstractHTTPJwtAuthenticator.java:103) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator.access$000(AbstractHTTPJwtAuthenticator.java:45) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator$1.run(AbstractHTTPJwtAuthenticator.java:85) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator$1.run(AbstractHTTPJwtAuthenticator.java:82) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at java.security.AccessController.doPrivileged(AccessController.java:312) [?:?]
        at com.amazon.dlic.auth.http.jwt.AbstractHTTPJwtAuthenticator.extractCredentials(AbstractHTTPJwtAuthenticator.java:82) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.extractCredentials(HTTPSamlAuthenticator.java:140) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.opendistroforelasticsearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:412) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.checkAndAuthenticateRequest(OpenDistroSecurityRestFilter.java:146) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter.access$000(OpenDistroSecurityRestFilter.java:63) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityRestFilter$1.handleRequest(OpenDistroSecurityRestFilter.java:93) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:222) [elasticsearch-7.4.2.jar:7.4.2]
        at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:295) [elasticsearch-7.4.2.jar:7.4.2]
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:166) [elasticsearch-7.4.2.jar:7.4.2]
        at com.amazon.opendistroforelasticsearch.security.ssl.http.netty.ValidatingDispatcher.dispatchRequest(ValidatingDispatcher.java:63) [opendistro_security-1.4.0.0.jar:1.4.0.0]
        at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:322) [elasticsearch-7.4.2.jar:7.4.2]
        at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:372) [elasticsearch-7.4.2.jar:7.4.2]
        at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:301) [elasticsearch-7.4.2.jar:7.4.2]
        at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:69) [transport-netty4-client-7.4.2.jar:7.4.2]
        at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:31) [transport-netty4-client-7.4.2.jar:7.4.2]
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.4.2.jar:7.4.2]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287) [netty-handler-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) [netty-handler-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) [netty-handler-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) [netty-handler-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) [netty-codec-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:597) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:551) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) [netty-transport-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) [netty-common-4.1.38.Final.jar:4.1.38.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.38.Final.jar:4.1.38.Final]
        at java.lang.Thread.run(Thread.java:830) [?:?]

```
