Environment Variables in pipelines.yaml?

Data Prepper
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

opensearchproject/data-prepper:2.1.0

Describe the issue:

I would like to use environment variables in pipelines.yaml to facilitate config changes for AWS ECS tasks.

Specifically the hosts array for opensearch sink and aws_sts_role_arn

Configuration:

I have tried ${env:FOO}, ${env.FOO}, ${FOO} but does not seem to work.

Relevant Logs or Screenshots:

2

@PeteW Thanks for your interest in using Data Prepper! Currently we have not supported injecting environment variables into our config file. You are welcome to open an feature request issue in our repository. Also you are welcome to provide more context on your ECS use case so that there might be workaround we can try to provide.

3

Thank you @qchea I will open a feature request issue with some more info and context there.

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.