# Running multiple pipelines using different YML files

**URL:** <https://forum.opensearch.org/t/running-multiple-pipelines-using-different-yml-files/12519>\
**Category:** Data Prepper\
**Created:** [February 15, 2023, 1:57pm UTC](https://forum.opensearch.org/t/running-multiple-pipelines-using-different-yml-files/12519 "2023-02-15T13:57:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![harshit\_patel](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@harshit\_patel](https://forum.opensearch.org/u/harshit_patel)\
**Post date:** [February 15, 2023, 1:57pm UTC](https://forum.opensearch.org/t/running-multiple-pipelines-using-different-yml-files/12519/1 "2023-02-15T13:57:33Z")

</div>

Hi,  
I am trying to create multiple pipelines in Data Prepper. I have the below YML file which creates 2 pipelines. But, is there a way to create two separate YML files, each for one pipeline, and run them both?

```auto
pipeline-1:
  workers: 2
  delay: 1000
  source:
    random:
  processor:
    - string_converter:
        upper_case: true
  sink:
    - opensearch:
        hosts: ["https://opensearch:9200"]
        insecure: true
        username: admin
        password: admin
        index: data-prepper-1

pipeline-2:
  workers: 2
  delay: 1000
  source:
    random:
  sink:
    - opensearch:
        hosts: ["https://opensearch:9200"]
        insecure: true
        username: admin
        password: admin
        index: data-prepper

```

---

<div class="post-metadata">

**Author:** ![shuklash](https://avatars.discourse-cdn.com/v4/letter/s/f17d59/32.png) [@shuklash](https://forum.opensearch.org/u/shuklash)\
**Post date:** [February 21, 2023, 5:24am UTC](https://forum.opensearch.org/t/running-multiple-pipelines-using-different-yml-files/12519/2 "2023-02-21T05:24:27Z")

</div>

Hi @harshit_patel

The example configuration you provided is the correct way to run multiple pipelines since Data Prepper is configured to have 1 Pipeline configuration YAML. You can checkout some log ingestion examples here: [data-prepper/log\_analytics.md at main · opensearch-project/data-prepper · GitHub](https://github.com/opensearch-project/data-prepper/blob/main/docs/log_analytics.md#pipeline-configuration)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [April 22, 2023, 5:24am UTC](https://forum.opensearch.org/t/running-multiple-pipelines-using-different-yml-files/12519/3 "2023-04-22T05:24:44Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.