OpenID connect integration with Opensearch

Hello @cwperks / All,

I am trying to integrate OpenID connect with Opensearch with fresh installation of single node cluster.

I was getting the error message as follows.

.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!

As I get to know that there is bug has been reported for the same it has been fixed as well…

[Backport 2.x] Fix legacy check in SecurityAdmin by opensearch-trigger-bot[bot] · Pull Request #2075 · opensearch-project/security (github.com)

This morning I tried to execute the securityadmin.sh script again with latest Opensearch version (2.2.1) and I am still getting that message…

Can you please assist me on this?

bash-4.2# ./securityadmin.sh -cd /usr/share/opensearch/config/opensearch-security -icl -nhnv \
>  -key /usr/share/opensearch/config/kirk-key.pem \
>  -cert /usr/share/opensearch/config/kirk.pem \
>  -cacert /usr/share/opensearch/config/root-ca.pem
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.2.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!
Populate config from /usr/share/opensearch/config/opensearch-security/
ERR: Seems /usr/share/opensearch/config/opensearch-security/config.yml is not in legacy format: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.opensearch.security.securityconf.impl.v7.ConfigV7$AuthcDomain` (although at least one Creator exists): no boolean/Boolean-argument constructor/factory method to deserialize from boolean value (true)

Here is my config.yaml

config:
  dynamic:
    authc:
      openid_auth_domain:
      http_enabled: true
      transport_enabled: true
      order: 0
      http_authenticator:
        type: openid
        challenge: false
        config:
          subject_key: preferred_username
          roles_key: admin_role
        openid_connect_idp:
          enable_ssl: true
          verify_hostnames: false
          openid_connect_url: https://idp-xxxxxxxxxx/.well-known/openid-configuration
          openid_connect_idp.pemtrustedcas_filepath: "/etc/ssl/certs/admin.pem"
      authentication_backend:
        type: noop

Here is my docker-compose file.

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - plugins.security.ssl.transport.pemcert_filepath=/etc/ssl/certs/admin.pem
      - plugins.security.ssl.transport.pemkey_filepath=/etc/ssl/certs/admin-key.pem
      - plugins.security.ssl.transport.pemtrustedcas_filepath=/etc/ssl/certs/root-ca.pem
      - plugins.security.ssl.transport.enforce_hostname_verification=false
      - plugins.security.ssl.http.enabled=true
      - plugins.security.ssl.http.pemcert_filepath=/etc/ssl/certs/admin.pem
      - plugins.security.ssl.http.pemkey_filepath=/etc/ssl/certs/admin-key.pem
      - plugins.security.ssl.http.pemtrustedcas_filepath=/etc/ssl/certs/root-ca.pem
      - plugins.security.authcz.admin_dn=CN=xyz.com,OU=xxxxxx,O=VW,L=wolfsburg,ST=xxxxxxxx,C=IN
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    # - "DISABLE_INSTALL_DEMO_CONFIG=true" # disables execution of install_demo_configuration.sh bundled with security plugin, which installs demo certificates and security configurations to OpenSearch
#     - "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
      - "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
      - ./config.yml:/usr/share/opensearch/config/opensearch-security/config.yml
      - /tmp/opensearch/:/etc/ssl/certs/
    ports:
      - 0.0.0.0:9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:latest
    container_name: opensearch-dashboards
    ports:
      - 0.0.0.0:5601:5601
    expose:
      - "5601"
    environment:
      - 'OPENSEARCH_HOSTS=["https://localhost:9200"]'
      - "opensearch_security.auth.type=openid"
      - "opensearch_security.openid.connect_url=https://xxxxxx.well-known/openid-configuration"
      - "opensearch_security.openid.client_id="XXXXXXXXXXXXXXXXXXX"
      - "opensearch_security.openid.client_secret=xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      - "opensearch_security.openid.base_redirect_url=https://localhost:5601"
      - "opensearch.requestHeadersWhitelist=[ authorization,securitytenant, security_tenant]"
      - "opensearch_security.openid.scope=openid profile email address phone offline_access"
      - "opensearch_security.openid.verify_hostnames:false"
      - "opensearch_security.openid.refresh_tokens:false"
#     - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
      - "SERVER_BASEPATH=/opensearch"
      - "SERVER_REWRITEBASEPATH=true"
    volumes:
      - /var/lib/docker/volumes/containers:/var/lib/docker/volumes/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - opensearch-net
volumes:
  opensearch-data1:

networks:
  opensearch-net:

Any help/suggestion would be highly appreciated…

Thanks.

Could it be that your code isnt indented correctly?

      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 5
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles...

@drBenway @cwperks

I tried by changing the indentations. However, no luck…

I am also having doubt here there according to the bug fixed recently for legacy issue. I should not get the following line while executing the securityadmin.sh.

Can you please provide your inputs here?

.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!
config:
  dynamic:
    authc:
      openid_auth_domain:
      http_enabled: true
      transport_enabled: true
      order: 0
      http_authenticator:
        type: openid
        challenge: false
        config:
          subject_key: preferred_username
          roles_key: admin_role
          openid_connect_idp:
            enable_ssl: true
            verify_hostnames: false
            openid_connect_url: https://xxxxxxxx/kums/.well-known/openid-configuration
            #openid_connect_idp.pemtrustedcas_filepath: "/tmp/admin.pem"
      authentication_backend:
        type: noop

Thanks.

Hello @cwperks,

I am referring to the below BUG and according to it I should not get the following error while executing the securityadmin.sh.

Please correct if I am wrong and kindly suggest me how to fix it and proceed with OpenID connect integration with Opensearch…

Thank you in advance.

.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!

Fix legacy check in SecurityAdmin by cwperks · Pull Request #2052 · opensearch-project/security (github.com)

Thanks.

Hey @sabil, the fix you linked to is scheduled to be included in the 2.3.0 release at the end of this week (9/15/22). This bug impacted 2.0, 2.1 and 2.2.

It sounds like the docker volume you are using may have been referenced elsewhere and already contains some data. Since this is a fresh installation, would it be ok to remove the existing volume or use a different named volume? i.e. instead of using opensearch-data1 you could name this something different so it creates a new volume, something like opensearch-data-node1.

See the documentation website on creating a cluster here: Docker - OpenSearch documentation - The documentation recommends using docker-compose down -v to remove volumes when spinning down a cluster.

Hello @cwperks,

Thank you for the information.

Okay sure, I will change the volume to opensearch-data-node1 and then will try after 15th Sep 2022.

Just FYI… I am using single node cluster for my opensearch.

Thanks.

Hi @sabil using a different volume is something you can do now. The bugfix linked above is for applying changes to an existing security index, but if you change the volume name it will create the security index on startup since it won’t exist on a new volume.

Hey @cwperks,

Thank you so much for your assistance… I have changed the volume name opensearch-data-new. Still getting the same error after executing the securityadmin.sh script.

Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.2.1
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!
Populate config from /usr/share/opensearch/config/opensearch-security/
ERR: Seems /usr/share/opensearch/config/opensearch-security/config.yml is not in legacy format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "http_enabled" (class org.opensearch.security.securityconf.impl.v7.ConfigV7), not marked as ignorable (one known property: "dynamic"])
 at [Source: (String)"{"openid_auth_domain":{"http_enabled":true,"transport_enabled":true,"order":0,"http_authenticator":{"type":"openid","challenge":false,"config":{"subject_key":"preferred_username","roles_key":"admin_role","openid_connect_idp":{"enable_ssl":true,"verify_hostnames":false,"openid_connect_url":"https://xyz.com/kums/.well-known/openid-configuration"}}}},"authentication_backend":{"type":"noop"}}"; line: 1, column: 43] (through reference chain: org.opensearch.security.securityconf.impl.SecurityDynamicConfiguration["openid_auth_domain"]->org.opensearch.security.securityconf.impl.v7.ConfigV7["http_enabled"])

Will try to use the docker image 2.2.3 tomorrow and see if it works with new release.

Thank you.

Hello @cwperks,

I have used 2.3.0 version and executed the securityadmin.sh… Still getting the errors

The Legacy index error has disappeared.

**Legacy index '.opendistro_security' (ES 6) detected (or forced). You should migrate the configuration!

However, still getting the following error message.

Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
OpenSearch Version: 2.3.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/opensearch/config/opensearch-security/
ERR: Seems /usr/share/opensearch/config/opensearch-security/config.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "http_enabled" (class org.opensearch.security.securityconf.impl.v7.ConfigV7), not marked as ignorable (one known property: "dynamic"])
 at [Source: (String)"{"openid_auth_domain":{"http_enabled":true,"transport_enab

Kindly suggest/assist on this…

Thanks.

Hey @sabil, make sure everything under openid_auth_domain: is indented to the right.

The nodes successfully startup for me with the configuration you posted and the section under openid_auth_domain indented.

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    authc:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: admin_role
            openid_connect_idp:
              enable_ssl: true
              verify_hostnames: false
              openid_connect_url: https://xxxxxxxx/kums/.well-known/openid-configuration
              #pemtrustedcas_filepath: /usr/share/opensearch/config/root-ca.pem
        authentication_backend:
          type: noop

Hey @cwperks,

Thank you for your assistance…

I have corrected the indentation. :slight_smile: Now when I try to execute the securityadmin.sh getting the following error message.

Security Admin v7
Will connect to localhost:9200 ... done
ERR: An unexpected ResponseException occured: method [GET], host [https://localhost:9200], URI [/_plugins/_security/whoami], status line [HTTP/1.1 403 Forbidden]
{"error":"GET /_plugins/_security/whoami API not whitelisted","status":"FORBIDDEN"}
Trace:
org.opensearch.client.ResponseException: method [GET], host [https://localhost:9200], URI [/_plugins/_security/whoami], status line [HTTP/1.1 403 Forbidden]
{"error":"GET /_plugins/_security/whoami API not whitelisted","status":"FORBIDDEN"}
        at org.opensearch.client.RestClient.convertResponse(RestClient.java:375)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:345)
        at org.opensearch.client.RestClient.performRequest(RestClient.java:320)
        at org.opensearch.security.tools.SecurityAdmin.execute(SecurityAdmin.java:462)
        at org.opensearch.security.tools.SecurityAdmin.main(SecurityAdmin.java:159)

I have found one Bug related to whiltelist.yml. I guess we need to keep both the files whitelist.yml and allowlist.yml until the opensearch 3.0 comes.

I have made the changes to whitelist.yml and allowlist.yml and added the following entry to it.

---
_meta:
  type: "allowlist"
  config_version: 2

# Description:
# enabled - feature flag.
# if enabled is false, all endpoints are accessible.
# if enabled is true, all users except the SuperAdmin can only submit the allowed requests to the specified endpoints.
# SuperAdmin can access all APIs.
# SuperAdmin is defined by the SuperAdmin certificate, which is configured with the opensearch.yml setting plugins.security.authcz.admin_dn:
# Refer to the example setting in opensearch.yml to learn more about configuring SuperAdmin.
#
# requests - map of allow listed endpoints and HTTP requests

#this name must be config
config:
  enabled: true
  requests:
    /_cluster/settings:
      - GET
    /_cat/nodes:
      - GET
    /_plugins/_security/whoami:
      - GET

Sorry to bother you again and again. I am very new to it. That’s the reason struggling a lot to do the integration of OpenID connect with Opensearch.

Any kind of help or suggestion is highly appreciated.

Thanks.

@sabil Make sure the subject line of the certificates you are using to run securityadmin with match the plugins.security.authcz.admin_dn line in your docker-compose to connect as the super admin and bypass the check.

Alternatively, as was mentioned earlier in this thread you can either delete the docker volume or use a different volume so that the security index is initialized on startup. You can use this option if you are starting from scratch to avoid needing to run the securityadmin script.

Hello @cwperks ,

Thank you for your response and suggestion.

I have included the value for plugins.security.authcz.admin_dn in docker-compose and I am getting following error.

Security Admin v7
Will connect to localhost:9200 ... done
Connected as "CN=kirk,OU=client,O=client,L=test,C=de"
ERR: "CN=kirk,OU=client,O=client,L=test,C=de" is not an admin user
Seems you use a client certificate but this one is not registered as admin_dn
Make sure opensearch.yml on all nodes contains:
plugins.security.authcz.admin_dn:
  - ""CN=kirk,OU=client,O=client,L=test,C=de""
bash-4.2#

Just wanted to know do we need to add admin_dn.yml as well for the security to enable.

One more thing I would like to know that after successful execution of securityadmin.sh. I should be able to login opensearch-dashbaord through OpenID connect or is there anything I need to do more in order to achieve this.

Thank you in advance for you all your help and support.

Thanks.

@sabil Any plugin.security... settings should be in the opensearch.yml file. See YAML files - OpenSearch documentation

One of the security settings in this file is plugins.security.authcz.admin_dn which takes a list of distinguished named. There is no admin_dn.yml, I think you may be mixing that up with nodes_dn.yml

If you create a custom custom_opensearch.yml you can mount it into the node container using

volumes:
   - /path/to/custom_opensearch.yml:/usr/share/opensearch/config/opensearch.yml

Hope that helps. If OIDC is successfully configured then you should be able to login to the dashboards through OpenID connect.

Again with securityadmin, it is not necessary to run securityadmin if you are starting a fresh cluster without any pre-existing data. Volumes in docker are a way to mount a local filesystem to that of a running container and are used to persist data in the event that a container is stopped and restarted. If you don’t reference a volume on the node in the docker-compose.yml file or remove the volume in between starting and stopping the cluster while troubleshooting the OIDC config then you will not need to run securityadmin.

Hey @cwperks,

Thank you so much for your kind responses and support. Really appreciate it. :slight_smile:

Now I have couple of queries. Hopefully the last ones :smiley:

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.cache.ttl_minutes: 60
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
node.max_local_storage_nodes: 3

In the above variables I can see file path for the certificate. Currently I am using the default certificate “Kirk” one. I guess those are already exist for testing.

  1. Should I just use the names kirk.pe, kirk.key and root-ca.pem something like that for the above parameter wherever it is asking for path.

  2. In Future if we wanted to use the our own certificates then it should from IDP. Am I right?

Meanwhile I will try to remove the volume and try to execute the docker-compose.yml.

Thanks,
Sabil.

I have not mounted the volume and use the above default values in docker-compose.yml.

However, still getting an error message as follows.

opensearch-node1         | [2022-09-16T16:03:56,561][INFO ][o.o.i.i.m.ISMTemplateService] [opensearch-node1] Failure experienced when migrating ISM Template and update ISM policies: {}
opensearch-node1         | [2022-09-16T16:03:56,604][INFO ][o.o.c.s.ClusterSettings  ] [opensearch-node1] updating [plugins.index_state_management.template_migration.control] from [0] to [-1]
opensearch-node1         | [2022-09-16T16:03:56,605][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node1         | [2022-09-16T16:03:56,608][INFO ][o.o.i.i.m.ISMTemplateService] [opensearch-node1] Successfully update template migration setting
opensearch-dashboards    | {"type":"log","@timestamp":"2022-09-16T16:04:49Z","tags":["error","opensearch","data"],"pid":1,"message":"[TimeoutError]: Request timed out"}
opensearch-dashboards    | {"type":"log","@timestamp":"2022-09-16T16:04:49Z","tags":["error","savedobjects-service"],"pid":1,"message":"Unable to retrieve version information from OpenSearch nodes."}
opensearch-node1         | [2022-09-16T16:04:56,553][INFO ][o.o.i.i.ManagedIndexCoordinator] [opensearch-node1] Cancel background move metadata process.
opensearch-node1         | [2022-09-16T16:04:56,554][INFO ][o.o.i.i.ManagedIndexCoordinator] [opensearch-node1] Performing move cluster state metadata.
opensearch-node1         | [2022-09-16T16:04:56,554][INFO ][o.o.i.i.MetadataService  ] [opensearch-node1] Move metadata has finished.

When I try to access the opensearch-dashboard getting an error that dashboard is not yet ready.

:expressionless: :neutral_face:

@sabil, kirk.pem is the demo admin when the demo certificates are installed. The paths there are relative to the $OPENSEARCH_HOME/config directory. In the docker images on docker hub, this path is /usr/share/opensearch/config on the nodes. When referencing the admin cert for running security admin you can use absolute paths or relative paths, relative to the directory securityadmin.sh is invoked from.

These are demo certificates and should not be used in production.

You can setup enabled TLS for when Opensearch connects to the IdP and can find the documentation for it here: OpenID Connect - OpenSearch documentation

Can you provide more output from opensearch-node1 for the error you are receiving on startup?

@cwperks Thank you for your response.

Yes, I am using full path of kirk.pem while running the securityadmin.sh script.

Following are the error messages I received.

Error message from Securityadmin.sh script execution.

        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: sun.security.validator.ValidatorException: Extended key usage does not permit use for TLS server authentication
        at java.base/sun.security.validator.EndEntityChecker.checkTLSServer(EndEntityChecker.java:305)
        at java.base/sun.security.validator.EndEntityChecker.check(EndEntityChecker.java:152)
        at java.base/sun.security.validator.Validator.validate(Validator.java:277)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:285)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1335)

Error message from docker logs.

WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.3.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.3.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2022-09-22T16:58:38,718][INFO ][o.o.n.Node               ] [opensearch-node1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-4046329902158116432, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2022-09-22T16:58:48,575][ERROR][o.o.s.c.ConfigurationRepository] [opensearch-node1] Cannot apply default config (this is maybe not an error!)
com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: while scanning for the next token
        at com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException.from(MarkedYAMLException.java:28) ~[jackson-dataformat-yaml-2.13.3.jar:2.13.3]

Docker-compose file. I am not using the volume now.

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:2.3.0
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - plugins.security.ssl.transport.pemcert_filepath=/usr/share/opensearch/config/kirk.pem
      - plugins.security.ssl.transport.pemkey_filepath=/usr/share/opensearch/config/kirk-key.pem
      - plugins.security.ssl.transport.pemtrustedcas_filepath=/usr/share/opensearch/config/root-ca.pem
      - plugins.security.ssl.transport.enforce_hostname_verification=false
      - plugins.security.ssl.http.enabled=true
      - plugins.security.ssl.http.pemcert_filepath=/usr/share/opensearch/config/kirk.pem
      - plugins.security.ssl.http.pemkey_filepath=/usr/share/opensearch/config/kirk-key.pem
      - plugins.security.ssl.http.pemtrustedcas_filepath=/usr/share/opensearch/config/root-ca.pem
      - "plugins.security.authcz.admin_dn=CN=kirk,OU=client,O=client,L=test,C=DE"
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - plugins.security.system_indices.enabled=true
      - plugins.security.system_indices.indices=[".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    # - "DISABLE_INSTALL_DEMO_CONFIG=true" # disables execution of install_demo_configuration.sh bundled with security plugin, which installs demo certificates and security configurations to OpenSearch
#     - "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
      - "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
            #- opensearch-new-data:/usr/share/opensearch/data
      - ./config.yml:/usr/share/opensearch/config/opensearch-security/config.yml
        #- /home/gnokq6j/opensearch/:/usr/share/opensearch/config/opensearch-security/
      - ./internal_users.yml:/usr/share/opensearch/config/opensearch-security/internal_users.yml
      - ./roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
      - ./tenants.yml:/usr/share/opensearch/config/opensearch-security/tenants.yml
      - ./roles.yml:/usr/share/opensearch/config/opensearch-security/roles.yml
      - ./action_groups.yml:/usr/share/opensearch/config/opensearch-security/action_groups.yml
      - ./nodes_dn.yml:/usr/share/opensearch/config/opensearch-security/nodes_dn.yml
      - ./whitelist.yml:/usr/share/opensearch/config/opensearch-security/whitelist.yml
      - ./allowlist.yml:/usr/share/opensearch/config/opensearch-security/allowlist.yml
        #- /home/gnokq6j/opensearch/admin.pem:/tmp/admin.pem
    ports:
      - 0.0.0.0:9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.3.0
    container_name: opensearch-dashboards
    ports:
      - 0.0.0.0:5601:5601
    expose:
      - "5601"
    environment:
      - 'OPENSEARCH_HOSTS=["https://localhost:9200"]'
      - "opensearch_security.auth.type=openid"
      - "opensearch_security.openid.connect_url=https://xyz.com/auth/realms/kums/.well-known/openid-configuration"
      - "opensearch_security.openid.client_id=xxxxxxxxxxxxxxxxxxxxxxxxxxx"
      - "opensearch_security.openid.client_secret=xxxxxxxxxxxxxxxxxxxxxxx"
      - "opensearch_security.openid.base_redirect_url=https://localhost:5601"
      - opensearch.ssl.verificationMode:none
        #- "opensearch.username:admin"
        #- "opensearch.password:admin"
      - "opensearch.requestHeadersWhitelist=[ authorization,securitytenant, security_tenant]"
      - "opensearch_security.openid.scope=openid profile email address phone offline_access"
      - "opensearch_security.openid.verify_hostnames:false"
      - "opensearch_security.openid.refresh_tokens:false"
#     - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
      - "SERVER_BASEPATH=/opensearch"
      - "SERVER_REWRITEBASEPATH=true"
    volumes:
      - /var/lib/docker/volumes/containers:/var/lib/docker/volumes/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - opensearch-net
        #volumes:
        #opensearch-new-data:

networks:
  opensearch-net:

config.yml

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    authc:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: admin_role
            openid_connect_idp:
              enable_ssl: true
              verify_hostnames: false
              openid_connect_url: https://xyz.com/auth/realms/kums/.well-known/openid-configuration
              pemtrustedcas_content: |-
                -----BEGIN CERTIFICATE-----
				Content of the kirk.pem key
                -----END CERTIFICATE-----
                # pemtrustedcas_filepath: /usr/share/opensearch/config/root-ca.pem
        authentication_backend:
          type: noop

Thanks.

Hi @nateynate. Could you move this case to the Security category?

@sabil The certificate used in the config.yml is incorrect.
The kirk certificate is an admin certificate and it is used to manage the security plugin configuration in the OpenSearch cluster.

The pemtrustedcas_content requires an IDP CA certificate (not a key) only when it’s self-signed.

What is your OpenID IDP?

@pablo Thank you for the information.

I am using our own IDP server for the open id connect authentication.

If I want use our own, do we need to have admin certificate and IDP CA certificate in place then only the authentication will work?

I was thinking to use default for testing purpose.

Kindly give some insights on the certificate… Thank you in advance! :slight_smile: