Geoip not working when parsing aws elb logs with logstash

OpenSearch
, ,
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
logstash-oss-with-opensearch-output-plugin:7.13.4
opensearch-dashboards:2.6.0
opensearch:2.6.0
filebeat 7.16.3

Describe the issue:
We were using FIlebeat aws module to send elb logs to elasticsearch. We want to move to Opensearch and during the testing we found out the Filbeat cannot connect to opensearch and Filebeat Oss doesn’t have the aws modules. So we send the Filebeat output to Logstash and thought we could forward the parsed fields from Logstash to Opensearch.

  1. In Logstash while processing the received message from Filebeat and parsing the client-ip using GeoIP we are getting the following error “_geoip_lookup_failure”. Following are the logs from Logstash startup.

[INFO ][logstash.filters.geoip ][main] DatabaseManager is not in classpath {:version=>“7.13.4”, :exception=>#<LoadError: no such file to load – /usr/share/logstash/x-pack/lib/filters/geoip/database_manager>}
[2023-07-05T17:08:53,323][INFO ][logstash.filters.geoip ][main] Using geoip database {:path=>“/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.1.3-java/vendor/GeoLite2-City.mmdb”, :healthy_database=>true}

Because the database_manager is missing, will the geoip processor not function in Logstash?

  1. We tried removing all the filters in Logstash to send all the fields received from Filebeat to be forwarded to Opensearch. That also didn’t work only the message field and other metadata fields were forwaded. So when Filebeat sends the fields to elasticsearch where does the parsing happen? Because when we put Filebeat output to a file, it has only metadata and message fields?

Does anyone have a solution for AWS ELB log processing with GeoIP? To be used with OpenSearch.

Configuration:

Filebeat Conf

filebeat.modules:

output.logstash:
hosts:

Logstash Pipeline

input {
beats {
port => 5044
} }
filter {

grok {
  match => { "message" => "%{DATA:type}\s+%{TIMESTAMP_ISO8601:time}\s+%{DATA:elb}\s+%{DATA:client_ip}\s+%{DATA:target}\s+%{BASE10NUM:request_processing_time}\s+%{DATA:target_processing_time}\s+%{BASE10NUM:response_processing_time}\s+%{BASE10NUM:elb_status_code}\s+%{DATA:target_status_code}\s+%{BASE10NUM:received_bytes}\s+%{BASE10NUM:sent_bytes}\s+\"%{DATA:request}\"\s+\"%{DATA:user_agent}\"\s+%{DATA:ssl_cipher}\s+%{DATA:ssl_protocol}\s+%{DATA:target_group_arn}\s+\"%{DATA:trace_id}\"\s+\"%{DATA:domain_name}\"\s+\"%{DATA:chosen_cert_arn}\"\s+%{DATA:matched_rule_priority}\s+%{TIMESTAMP_ISO8601:request_creation_time}\s+\"%{DATA:actions_executed}\"\s+\"%{DATA:redirect_url}\"\s+\"%{DATA:error_reason}\"\s+\"%{DATA:target_list}\"\s+\"%{DATA:target_status_code_list}\"\s+\"%{DATA:classification}\"\s+\"%{DATA:classification_reason}\"" }
  }

useragent {
source=> “user_agent”
prefix=> “browser”
}
geoip {
source => “client_ip” }
}
output {
opensearch {
hosts =>

Relevant Logs or Screenshots:

Logstash logs

[0] “forwarded”,
[1] “beats_input_codec_plain_applied”,
[2] “_geoip_lookup_failure”

2

Do you have the loglines after grok that you are trying to do a geoip for ? Are they valid non RFC1918 IP addresses?

3

@jasonrojas

useragent {
source=> “user_agent”
prefix=> “browser”
}
geoip {
source => “client_ip” }
}

Are these the lines you asked about?

4

No, thats your config. what does the end result log line look like after grok parsing. You may be getting the geoip exception because the field may contain something its not expecting.

5

@jasonrojas thanks a lot, actually the port number also came with IP in the client_ip filed.
Added the following before geoip to fix it.

mutate {
gsub => [ “client_ip”, “:\d+”, “” ]
}

1 Like