Hello!
Please help with running logstash on linux (RedHat) machine.
I have a running instance of Opensearch 2.11.1 on http://my_server_IP:9200
I downloaded logstash-oss-7.12.1-linux-x86_64.tar.gz and unpacked it to my applications folder (the same server as opensearch)
In opensearch.yml I have set:
plugins.security.disabled: true
plugins.security.ssl.http.enabled: false
My logstash.conf is as follows:
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["http://my_server_IP:9200"]
index => "logs-localhost_host-app1-%{+yyyy-MM-dd}"
ssl_certificate_verification => false
user => "my_admin"
password => "my_password"
}
}
When I try to run logstash from command:
/my_apps_folder/logstash-7.12.1/bin/logstash -f /my_apps_folder/logstash-7.12.1/config/logstash.conf
I get and an error:
Using bundled JDK: /my_apps_folder/logstash-7.12.1/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /my_apps_folder/logstash-7.12.1/logs which is now configured via log4j2.properties
[2025-02-17T18:30:03,207][INFO ][logstash.runner ] Log4j configuration path used is: /my_apps_folder/logstash-7.12.1/config/log4j2.properties
[2025-02-17T18:30:03,216][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.12.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9 on 11.0.10+9 +indy +jit [linux-x86_64]"}
[2025-02-17T18:30:03,445][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2025-02-17T18:30:03,857][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2025-02-17T18:30:04,396][INFO ][org.reflections.Reflections] Reflections took 21 ms to scan 1 urls, producing 23 keys and 47 values
[2025-02-17T18:30:04,912][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://my_admin:xxxxxx@my_server_IP/:9200]}}
[2025-02-17T18:30:05,032][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://my_admin:xxxxxx@my_server_IP/:9200"}
[2025-02-17T18:30:05,066][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `[]' for nil:NilClass>, :backtrace=>["/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:470:in `get_es_version'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:273:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:265:in `healthcheck!'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:367:in `update_urls'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:83:in `update_initial_urls'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:77:in `start'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client.rb:338:in `build_pool'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client.rb:64:in `initialize'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch.rb:270:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:585:in `maybe_setup_out_plugins'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/my_apps_folder/logstash-7.12.1/config/logstash.conf"], :thread=>"#<Thread:0x327baa32 run>"}
[2025-02-17T18:30:05,068][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2025-02-17T18:30:05,081][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2025-02-17T18:30:05,142][INFO ][logstash.runner ] Logstash shut down.
[2025-02-17T18:30:05,147][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at my_apps_folder.logstash_minus_7_dot_12_dot_1.lib.bootstrap.environment.<main>(/my_apps_folder/logstash-7.12.1/lib/bootstrap/environment.rb:89) ~[?:?]
When I run:
./logstash --config.debug --log.level debug -f /my_apps_folder/logstash-7.12.1/config/logstash.conf
...
[2025-02-17T18:50:23,412][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
...
[2025-02-17T18:50:23,708][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config_file=>"/my_apps_folder/logstash-7.12.1/config/logstash.conf"}
...
[2025-02-17T18:50:25,321][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://my_admin:xxxxxx@my_server_IP/:9200"}
[2025-02-17T18:50:25,353][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `[]' for nil:NilClass>, :backtrace=>["/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:470:in `get_es_version'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:273:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:265:in `healthcheck!'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:367:in `update_urls'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:83:in `update_initial_urls'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:77:in `start'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client.rb:338:in `build_pool'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client.rb:64:in `initialize'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'", "/my_apps_folder/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.8.6-java/lib/logstash/outputs/elasticsearch.rb:270:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:585:in `maybe_setup_out_plugins'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/my_apps_folder/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/my_apps_folder/logstash-7.12.1/config/logstash.conf"], :thread=>"#<Thread:0x927c843 run>"}
[2025-02-17T18:50:25,354][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2025-02-17T18:50:25,372][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2025-02-17T18:50:25,393][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2025-02-17T18:50:25,397][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2025-02-17T18:50:25,398][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2025-02-17T18:50:25,398][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2025-02-17T18:50:25,400][DEBUG][logstash.agent ] Shutting down all pipelines {:pipelines_count=>0}
[2025-02-17T18:50:25,401][DEBUG][logstash.agent ] Converging pipelines state {:actions_count=>0}
[2025-02-17T18:50:25,425][INFO ][logstash.runner ] Logstash shut down.
[2025-02-17T18:50:25,430][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at my_apps_folder.logstash_minus_7_dot_12_dot_1.lib.bootstrap.environment.<main>(/my_apps_folder/logstash-7.12.1/lib/bootstrap/environment.rb:89) ~[?:?]