Health check problem: no API cause 200 ok

Hi.
I downloaded the data-prepper docker image (latest version):
docker pull opensearchproject/data-prepper:latest
If i run the container, this works. In particular with the following logs:

data-prepper    | 2022-12-12T16:51:35,428 [main] INFO  org.opensearch.dataprepper.plugins.source.oteltrace.OTelTraceSource - Started otel_trace_source on port 21890.
data-prepper    | 2022-12-12T16:51:35,438 [main] INFO  org.opensearch.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900

If I do a GET to http://localhost:21890 I get error 404.
My problem is the health check. I took this docker to AWS to try and make some configurations, but unfortunately the health check process on AWS always fails.
I found this bug on the official GitHub page but unfortunately I couldn’t find a correct path that causes the API to go 200 OK (in order to have a valid healthcheck).

Do I need to add some Docker configuration to fix this problem?

Hi @czanacchi,

Please see this README regarding otel trace source and its health check. You may need to adjust your config file accordingly. Please let me know if you need more assistance, I’d be happy to help!

Hi @ddpowers ,
Even editing the pipelines.yaml configuration file doesn’t solve the problem.
The problem I have is the following:

I’m working on AWS ECS and I’m trying to set up a task with data-prepper Docker image inside, but the associated AWS Target Group healthcheck is always indicated as unhealthy.
This happens because all HTTP calls to the container (which are made automatically by AWS) result in a 404 or 401.

How can I edit the files so that I can get a correct healtcheck HTTP call that results in 200 OK?

Can you please share your config file?

This is the file called data-prepper-config.yaml:

ssl: false

And this is the pipelines.yaml file:

entry-pipeline:
  delay: "100"
  source:
    otel_trace_source:
      ssl: false
  buffer:
    bounded_blocking:
      buffer_size: 10240
      batch_size: 160
  sink:
    - pipeline:
        name: "raw-pipeline"
    - pipeline:
        name: "service-map-pipeline"
raw-pipeline:
  source:
    pipeline:
      name: "entry-pipeline"
  buffer:
    bounded_blocking:
      buffer_size: 10240
      batch_size: 160
  processor:
    - otel_trace_raw:
  sink:
    - opensearch:
        hosts: [ "URL_AT_MY_AWS_OPENSEARCH" ]
        # IAM signing
        username: "my-AWS-user"
        password: "my-AWS-password"
        aws_region: AWS-region
        index_type: trace-analytics-raw
service-map-pipeline:
  delay: "100"
  source:
    pipeline:
      name: "entry-pipeline"
  buffer:
    bounded_blocking:
      buffer_size: 10240
      batch_size: 160
  processor:
    - service_map_stateful:
  sink:
    - opensearch:
        hosts: [ "URL_AT_MY_AWS_OPENSEARCH" ]
        # IAM signing
        username: "my-AWS-user"
        password: "my-AWS-password"
        aws_region: AWS-region
        index_type: trace-analytics-service-map

With this configuration file, if I try to deploy on AWS, the health check of the Target group associated with the data prepper task fails.

@czanacchi,

Please enable unframed_requests. This should enable health checks

Hi @ddpowers. In which part of the file pipelines.yaml should I insert this unframed_requests directive? What kind of value it takes (string, boolean, etc…)?

Hi @czanacchi,
unframed_requests is an option of boolean type for otel_trace_source in the pipelines.yaml file. It will allow health check through HTTP.
The table in this reference has details.