# Data-prepper example log-ingestion not working

**URL:** <https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381>\
**Category:** Data Prepper\
**Created:** [August 15, 2025, 11:40am UTC](https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381 "2025-08-15T11:40:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bernd](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bernd](https://forum.opensearch.org/u/bernd)\
**Post date:** [August 15, 2025, 11:40am UTC](https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381/1 "2025-08-15T11:40:27Z")

</div>

**Versions:**

Latest version

**Describe the issue** :

I tried the example of log-ingestion, downloaded all yaml files, startet the first docker (docker-compose.yaml), logged in to [http://127.0.0.1:5601/](http://127.0.0.1:5601/) and all looks fine.

Next step is loading an Apache log file as “test.log” by starting a second docker (docker-compose-dataprepper.yaml).

It starts an the last line is “… Started http source on port 2021…”.

No errors so far, but it doesn’t start ingesting the test.log file.

At the Dashboard I can create an index pattern and see “apache\_logs” for selection.

After creating an index pattern and going to Discover there is only an empty index. Nothing has been ingested from test.log.

Any idea why?

Regards Bernd

**Configuration** :

**Relevant Logs or Screenshots** :

---

<div class="post-metadata">

**Author:** ![Leeroy](https://avatars.discourse-cdn.com/v4/letter/l/eb9ed0/32.png) [@Leeroy](https://forum.opensearch.org/u/Leeroy)\
**Post date:** [August 15, 2025, 11:50am UTC](https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381/2 "2025-08-15T11:50:38Z")

</div>

Hey @bernd ,

please share all your config files so we can get a better picture to what might be going wrong.

Leeroy.

---

<div class="post-metadata">

**Author:** ![bernd](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bernd](https://forum.opensearch.org/u/bernd)\
**Post date:** [August 15, 2025, 12:20pm UTC](https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381/3 "2025-08-15T12:20:48Z")

</div>

docker-compose.yaml

```auto
version: '3'
services:
  fluent-bit:
    container_name: fluent-bit
    image: docker.io/fluent/fluent-bit
    volumes:
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
      - ./test.log:/var/log/test.log
    networks:
      - opensearch-net
  opensearch:
    container_name: opensearch
    image: docker.io/opensearchproject/opensearch:latest
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - "OPENSEARCH_INITIAL_ADMIN_PASSWORD=Developer@123"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  dashboards:
    container_name: opensearch-dashboards
    image: docker.io/opensearchproject/opensearch-dashboards:latest
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch:9200"]'
    depends_on:
      - opensearch
    networks:
      - opensearch-net

networks:
  opensearch-net:

```

docker-compose-dataprepper.yaml

```auto
version: '3.7'
services:
  data-prepper:
    container_name: data-prepper
    image: opensearchproject/data-prepper:latest
    volumes:
      - ./log_pipeline.yaml:/usr/share/data-prepper/pipelines/log_pipeline.yaml
      - ./data-prepper-config.yaml:/usr/share/data-prepper/config/data-prepper-config.yaml
    ports:
      - 2021:2021
    networks:
      - opensearch-net

networks:
  opensearch-net:
    driver: bridge

```

log\_pipeline.yaml

```auto
log-pipeline:
  source:
    http:
      ssl: false
  processor:
    - grok:
        match:
          log: ["%{COMBINEDAPACHELOG}"]
  sink:
    - opensearch:
        hosts: ["https://opensearch:9200"]
        insecure: true
        username: admin
        password: Developer@123
        index: apache_logs

```

data-prepper-config.yaml

```auto
ssl: false
serverPort: 4900
authentication:
  http_basic:
    username: admin
    password: Developer@123

```

fluent-bit.conf

```auto
[INPUT]
  name tail
  refresh_interval 5
  path /var/log/test.log
  read_from_head true

[OUTPUT]
  Name http
  Match *
  Host data-prepper
  Port 2021
  URI /log/ingest
  Format json
  HTTP_User admin
  HTTP_Passwd Developer@123

```

First Docker:

```auto
docker compose --project-name data-prepper -f docker-compose.yaml up

```

Second Docker:

```auto
docker compose --project-name data-prepper -f docker-compose-dataprepper.yaml up

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [October 14, 2025, 12:21pm UTC](https://forum.opensearch.org/t/data-prepper-example-log-ingestion-not-working/26381/4 "2025-10-14T12:21:21Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.