Using environment variables for OpenID Connect URL and secret

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
2.18

Describe the issue:
I need to use environment variables for the security settings related to OpenID Connect authentication (URL, client id and secret) in a docker compose file.

I’ve tried different combinations but the only one that works is to provide the literal value in the opensearch_dashboards.yml file.

I tried adding environmetns section in the docker compose file:

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.18.0
    environment:
      ..
      # using variable from .env file:
      OPENSEARCH_SECURITY_OPENID_CONNECT_URL: "${OPENSEARCH_OPENID_CONNECT_URL}"

      # using lower case variable:
      opensearch_security.openid.connect_url: '${OPENSEARCH_SECURITY_OPENID_CONNECT_URL}'

The opensearch_dashboards.yml file also is not taking variables. I tried:

opensearch_security.openid.connect_url: "${OPENSEARCH_OPENID_CONNECT_URL}"

I also tried using literal values but none are taken. The error always says:

2024-12-18T23:21:11.658631690Z {“type”:“log”,“@timestamp”:“2024-12-18T23:21:11Z”,“tags”:[“fatal”,“root”],“pid”:1,“message”:"Error: Failed when trying to obtain the endpoints from your IdP\n at OpenIdAuthentication.init (/usr/share/opensearch-dashboards/plugins/securityDashboards/server/auth/types/openid/openid_auth.ts:126:13)\n at MultipleAuthentication.init

The only thing that works is using literal values in opensearch_dashboards.yml file. Is there no way to use environment variables for these settings?

2

@dmossakowski According to my tests, OpenSearch Dashboards will ignore environmental settings for security plugin. It must be placed directly in the opensearch_dashboards.yml file.

3

After many tests I was able to use environment variables for all services: open search nodes, dashboards and data prepper. In all cases I load the variables from local .env file and so my docker compose file has:

services:
  opensearch-node1:
    env_file:
      - .env
    image: opensearchproject/opensearch:2.18.0

config.yml

In this file you need to specify variables with env prepended like this:

      oidc_auth_domain:
        description: "Authenticate via OIDC (OpenID Connect)"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: "openid"
          challenge: true
          config:
            subject_key: "preferred_username"
            roles_key: "roles"
            openid_connect_url: ${env.OPENSEARCH_SECURITY_OPENID_CONNECT_URL}
            client_id: ${env.OPENSEARCH_SECURITY_OPENID_CLIENT_ID}
            client_secret: ${env.OPENSEARCH_SECURITY_OPENID_CLIENT_SECRET}

opensearch_dashboards.yml

Here the variables seem to work normally:

opensearch_security.auth.type: ["basicauth","openid"]
opensearch_security.openid.connect_url: ${OPENSEARCH_SECURITY_OPENID_CONNECT_URL}
opensearch_security.openid.client_id: ${OPENSEARCH_SECURITY_OPENID_CLIENT_ID}
opensearch_security.openid.client_secret: ${OPENSEARCH_SECURITY_OPENID_CLIENT_SECRET}
opensearch_security.openid.base_redirect_url: ${OPENSEARCH_SECURITY_OPENID_BASE_REDIRECT_URL}

data prepper

Data prepper config doesn’t support environment variables so I used envsubst. Locally I only have a config file named data-prepper-pipelines.yaml.template and in docker I create a shared volume so the template file is processed and put on the shared volume where then it is read by data prepper service.

data prepper pipeline template file:

  sink:
    - opensearch:
        hosts: ["https://opensearch-node1:9200"]
        username: admin
        password: ${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
        index_type: trace-analytics-service-map
        insecure: true
#     - stdout:

docker compose:
(I used envsubst from nginx image because I already use nginx for another service)

  envsubst-service:
    image: nginx
    env_file:
      - .env
    command: /bin/bash -c "envsubst < /templates/data-prepper-pipelines.yaml.template > /output/pipelines.yaml"
    volumes:
      - ./data-prepper-pipelines.yaml.template:/templates/data-prepper-pipelines.yaml.template
      - shared-data-prepper:/output
    networks:
      - opensearch-net


  data-prepper:
    env_file:
      - .env
    image: opensearchproject/data-prepper:2.10.2
    ports:
      - 21890:21890
      - 21891:21891
    expose:
      - "21890"
      - "21891"
    volumes:
      - shared-data-prepper:/usr/share/data-prepper/pipelines
     

    networks:
      - opensearch-net
    depends_on:
      - envsubst-service
      - opensearch-dashboards

Hopefully this helps someone.

2 Likes