# Filebeat not pussing every logs to elasticsearch there is discrepancy in the logs

**URL:** <https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485>\
**Category:** Open Source Elasticsearch and Kibana\
**Tags:** discuss, troubleshoot\
**Created:** [July 24, 2024, 6:18am UTC](https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485 "2024-07-24T06:18:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nandeeshb](https://avatars.discourse-cdn.com/v4/letter/n/ecccb3/32.png) [@Nandeeshb](https://forum.opensearch.org/u/Nandeeshb)\
**Post date:** [July 24, 2024, 6:18am UTC](https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485/1 "2024-07-24T06:18:49Z")

</div>

Hi Team,

Really need a help on this , as Im not able to figure out whats the exact issue.

Filebeat version: 8.3.3  
Elastic search version installed on vms: 8.12.0

We are using filebeat to push the logs to elastic search based on the tags with namespace in kubernetes.

Everytime we are missing the logs , only few logs are pushing and not all the logs.

What might be the issue? Please help me on this…

below is the config file which we are using.

filebeat.yml: |-  
filebeat.inputs:  
- type: container  
paths:  
- /var/log/containers/_uat_  
tags: [“uat”]  
processors:  
- add\_kubernetes\_metadata:  
host: ${NODE\_NAME}  
matchers:  
- logs\_path:  
logs\_path: “/var/log/containers/”  
- type: container  
paths:  
- /var/log/containers/_qa_  
tags: [“qa”]  
processors:  
- add\_kubernetes\_metadata:  
host: ${NODE\_NAME}  
matchers:  
- logs\_path:  
logs\_path: “/var/log/containers/”  
- type: container  
paths:  
- /var/log/containers/_dev_  
tags: [“dev”]  
processors:  
- add\_kubernetes\_metadata:  
host: ${NODE\_NAME}  
matchers:  
- logs\_path:  
logs\_path: “/var/log/containers/”

```
processors:
  - add_cloud_metadata:
  - add_host_metadata:

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
  hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:443}']
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}
  protocol: "https"
  indices:
  - index: "uat-logs-%{+yyyy.MM.dd}"
    when.contains:
      tags: "uat"
  - index: "qa-logs-%{+yyyy.MM.dd}"
    when.contains:
      tags: "qa"
  - index: "dev-logs-%{+yyyy.MM.dd}"
    when.contains:
      tags: "dev"

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/3/33547ea01a5b12dcca2958411d3edd97ae2ea8c1.png) [@system](https://forum.opensearch.org/u/system)\
**Post date:** [September 22, 2024, 6:19am UTC](https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485/2 "2024-09-22T06:19:23Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.