Hi,
I am trying to use logstash and fluentd in two different Instances to test logs forwarding. However I am get in to some issues which are related to SSL certificates. I could see the logs are receiving to the OpenDistro node in a tcpdump, but nothing has been inserted to the Elasticsearch.
should I need to use https instead of http in all over the configurations? Can you please share a little guide to fix this configuration issue?
Thank you
Luke
Is you issue resolved, I have same issue with you. If you fixed your issue, please post it in this topic. thanks.
Hi,
Yes, I managed to resolve the issue. Try the following config in your td-agent.conf
<match *.**>
@type elasticsearch
hosts https://admin:admin@localhost:9200
scheme https
ssl_verify false
ssl_version TLSv1_2
logstash_format true
logstash_prefix fluentd
enable_ilm true
index_date_pattern “now/m{yyyy.mm}”
flush_interval 10s
Cheers,
Luke.
Hi Luke,
Can you share your logstash config and Filebeat config like below?
Logstash config:
input {
file {
path => “/var/log/messages”
start_position => “beginning”
}
}
filter {
}
output {
elasticsearch {
hosts => [“https://192.168.4.231:9200”]
index => “messages-%{+YYYY-MM-dd}”
user => admin
password => admin
ssl => true
ssl_certificate_verification => false
ilm_enabled => false
cacert => “/etc/logstash/root-ca.pem”
}
Filebeat Config:
filebeat.config.modules:
enabled: true
path: ${path.config}/modules.d/*.yml
setup.ilm.overwrite: true
setup.ilm.enabled: false
output.elasticsearch:
ssl.verification_mode: none
hosts: [“localhost”]
username: “admin”
password: “admin”
setup.kibana:
host: “localhost”
logging.level: debug
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat
keepfiles: 7
permissions: 0644
Best Regards,
Stone Liu
@luke_devon
Sorry to spam some older posts - but wanted to mention for fluentd - please look to the newly released fluent-plugin-opensearch
link: GitHub - fluent/fluent-plugin-opensearch: OpenSearch Plugin for Fluentd
