Filebeat doesn't work correclty

General Feedback
1

I want to read /var/log/syslog.log with filebeat 7.12.1 and send to logstash 8.19 and then put on opensearch to see on opensearch dashboard, filebeat said that is conected to logstash but can´t read syslog.log. Anyone know why.

I use opensearch 3.5.

2

@lzamorano Thank you for the question, Can you provide your configuration please.

I tried locally using docker-compose with the following configuration at received the expected documents in opensearch:

filebeat.yml

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/logs.txt 
    fields:
      log_type: syslog
    fields_under_root: true



output.logstash:
  hosts: ["logstash:5044"]

# Logging
logging.level: info
logging.to_files: false
logging.files:
  path: log
  name: filebeat
  keepfiles: 7
  permissions: 0644

Logstash:

input {
  beats {
    port => 5044
  }
}

filter {
  # Optional: parse syslog format
  if [fields][log_type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  opensearch {
    hosts           => ["http://opensearch:9200"]
    index           => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    user            => "admin"
    password        => "password"  # Match OPENSEARCH_INITIAL_ADMIN_PASSWORD above
    ssl             => false
    manage_template       => false
    ssl_certificate_verification => false   # Set to true in production with proper certs
  }
}