I’m running a development set of servers (not production). I have an ELK server, a web server, and a reverse proxy server. Data from the web server and the reverse proxy have successfully been sent into Logstash on the ELK server. Kibana’s filebeat discovery does show the hostname of all three servers. So I know the configuration is correct.
Running the following command:
sudo filebeat -e -c /etc/filebeat/filebeat.yml test output on both remote servers (the web server and the reverse proxy server) has the following response:
logstash: 192.168.1.6:5044... connection... parse host... OK dns lookup... OK addresses: 192.168.1.6 dial up... OK TLS... WARN secure connection disabled talk to server... OK
Running the command to see the status of the filebeat service (on both remote servers) reflects the service is active and has a recent timestamp of log data that is being collected. I did this a few times over a span of 15 minutes, and I can continuously see new time stamps of log collection by file beat on the remote servers.
But for some reason, the only time filebeat actually sends data to Logstash on the ELK server is upon reboot of the web server and reverse proxy server. And only just once. Filebeat data is not a continuous stream of data into Logstash.
So my question is: How is it possible that testing filebeat output shows a successful connection, that filebeat data is successfully seen in elasticsearch, and that Kibana discovery has valid data for my servers? But, filebeat is not sending continuously into logstash.
Oh, one more thing. The ELK server does, in fact, have a continuous stream of filebeat data from itself (the ELK server’s filebeat is continuously streaming data to logstash). It’s the other servers that are not feeding continuously.
Running Elasticstack version 7.17.13 on Ubuntu servers. ELK server is Ubuntu server 20.04 (minimal), web server is Ubuntu server 20.04 (minimal), and the reverse proxy server is Ubuntu server 22.04 (minimal). All are on the same subnet. Running as virtual servers in an ESXi VMware platform.
I’ve tried purging filebeat from the web server and the reverse proxy server and ensuring the filebeat configurations are deleted.
Then, reinstalling filebeat and putting in a fresh configuration to send all data to logstash on the ELK server and test the filebeat outputs.
Everything looks OK, but I see only one stream of data sent to logstash immediately upon rebooting the servers! Would like to see a continuous stream of filebeat data from other servers. Ughhhhh!