Using environment variables for OpenID Connect URL and secret

OpenSearch Dashboards
3

After many tests I was able to use environment variables for all services: open search nodes, dashboards and data prepper. In all cases I load the variables from local .env file and so my docker compose file has:

services:
  opensearch-node1:
    env_file:
      - .env
    image: opensearchproject/opensearch:2.18.0

config.yml

In this file you need to specify variables with env prepended like this:

      oidc_auth_domain:
        description: "Authenticate via OIDC (OpenID Connect)"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: "openid"
          challenge: true
          config:
            subject_key: "preferred_username"
            roles_key: "roles"
            openid_connect_url: ${env.OPENSEARCH_SECURITY_OPENID_CONNECT_URL}
            client_id: ${env.OPENSEARCH_SECURITY_OPENID_CLIENT_ID}
            client_secret: ${env.OPENSEARCH_SECURITY_OPENID_CLIENT_SECRET}

opensearch_dashboards.yml

Here the variables seem to work normally:

opensearch_security.auth.type: ["basicauth","openid"]
opensearch_security.openid.connect_url: ${OPENSEARCH_SECURITY_OPENID_CONNECT_URL}
opensearch_security.openid.client_id: ${OPENSEARCH_SECURITY_OPENID_CLIENT_ID}
opensearch_security.openid.client_secret: ${OPENSEARCH_SECURITY_OPENID_CLIENT_SECRET}
opensearch_security.openid.base_redirect_url: ${OPENSEARCH_SECURITY_OPENID_BASE_REDIRECT_URL}

data prepper

Data prepper config doesn’t support environment variables so I used envsubst. Locally I only have a config file named data-prepper-pipelines.yaml.template and in docker I create a shared volume so the template file is processed and put on the shared volume where then it is read by data prepper service.

data prepper pipeline template file:

  sink:
    - opensearch:
        hosts: ["https://opensearch-node1:9200"]
        username: admin
        password: ${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
        index_type: trace-analytics-service-map
        insecure: true
#     - stdout:

docker compose:
(I used envsubst from nginx image because I already use nginx for another service)

  envsubst-service:
    image: nginx
    env_file:
      - .env
    command: /bin/bash -c "envsubst < /templates/data-prepper-pipelines.yaml.template > /output/pipelines.yaml"
    volumes:
      - ./data-prepper-pipelines.yaml.template:/templates/data-prepper-pipelines.yaml.template
      - shared-data-prepper:/output
    networks:
      - opensearch-net


  data-prepper:
    env_file:
      - .env
    image: opensearchproject/data-prepper:2.10.2
    ports:
      - 21890:21890
      - 21891:21891
    expose:
      - "21890"
      - "21891"
    volumes:
      - shared-data-prepper:/usr/share/data-prepper/pipelines
     

    networks:
      - opensearch-net
    depends_on:
      - envsubst-service
      - opensearch-dashboards

Hopefully this helps someone.

2 Likes