Can't you automatically turn off the bin/data-prepper?

Data Prepper
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch-data-prepper-2.7.0-SNAPSHOT-linux-x64

Describe the issue:

There are 4 pieces of data under the studio folder. These 4 pieces of data are stored once a month, and I’m going to use crontab to turn the layout around.
If I access the folder with the for statement and run bin/data-prepper, it won’t turn off, so I can’t move on to the next data. Is there a way to automatically turn it off when data is registered in OpenSearch??

Configuration:

Relevant Logs or Screenshots:

version: 2
sample-pipeline:
  workers: 8
  delay: 100
  source:
    file:
      path: "/mnt/storage1/202403/student3.csv"
      codec:
        csv:
  buffer:
    bounded_blocking:
      buffer_size: 10000000
      batch_size: 1000000
  sink:
    - opensearch:
        hosts: ["https://localhost:9200"]
        username: "admin"
        password: "admin"
        index: "student3_test"
        insecure: true
2

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.