I have been working on this project, and my logstash configuration file works fine. Now I am trying to implement the same thing on a system that is running opendistro/elk stack.
I have tried to fix most of the errors, but now I am receiving this and not sure how to proceed further
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/tmp/jruby-27507/jruby3696319005608397921jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2020-10-22 18:15:29.196 [main] runner - Starting Logstash {"logstash.version"=>"7.9.2", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10-LTS on 11.0.8+10-LTS +indy +jit [linux-x86_64]"}
[WARN ] 2020-10-22 18:15:29.917 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-10-22 18:15:34.177 [Converge PipelineAction::Create<main>] Reflections - Reflections took 56 ms to scan 1 urls, producing 22 keys and 45 values
[WARN ] 2020-10-22 18:15:35.246 [Converge PipelineAction::Create<main>] elasticsearch - You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"document_type", :plugin=><LogStash::Outputs::ElasticSearch ssl_certificate_verification=>false, password=><password>, hosts=>[https://localhost:9200], ilm_enabled=>"false", index=>"kcpvxbeattest-%{+YYYY.MM.dd}", manage_template=>false, id=>"abcf8d52a697c223d3765dc35e8371201acfa7c8434f2485297dac6a743a33a3", user=>"admin", ssl=>true, document_type=>"logs", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_bd36febb-370d-4b8e-94d8-3984de1dcf25", enable_metric=>true, charset=>"UTF-8">, workers=>1, template_overwrite=>false, doc_as_upsert=>false, script_type=>"inline", script_lang=>"painless", script_var_name=>"event", scripted_upsert=>false, retry_initial_interval=>2, retry_max_interval=>64, retry_on_conflict=>1, ilm_pattern=>"{now/d}-000001", ilm_policy=>"logstash-policy", ecs_compatibility=>:disabled, action=>"index", sniffing=>false, sniffing_delay=>5, timeout=>60, pool_max=>1000, pool_max_per_route=>100, resurrect_delay=>5, validate_after_inactivity=>10000, http_compression=>false>}
[WARN ] 2020-10-22 18:15:35.896 [[main]-pipeline-manager] elasticsearch - ** WARNING ** Detected UNSAFE options in elasticsearch output configuration!
** WARNING ** You have enabled encryption but DISABLED certificate verification.
** WARNING ** To make sure your data is secure change :ssl_certificate_verification to true
[INFO ] 2020-10-22 18:15:36.628 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://admin:xxxxxx@localhost:9200/]}}
[WARN ] 2020-10-22 18:15:37.387 [[main]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"https://admin:xxxxxx@localhost:9200/"}
[INFO ] 2020-10-22 18:15:37.734 [[main]-pipeline-manager] elasticsearch - ES Output version determined {:es_version=>7}
[WARN ] 2020-10-22 18:15:37.740 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[INFO ] 2020-10-22 18:15:37.784 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://localhost:9200"]}
[ERROR] 2020-10-22 18:15:38.299 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[INFO ] 2020-10-22 18:15:38.873 [LogStash::Runner] runner - Logstash shut down.
output {
elasticsearch {
hosts => ["https://localhost:9200"]
index => "kcpvxbeattest-%{+YYYY.MM.dd}"
user => "admin"
password => "admin"
ssl => true
ssl_certificate_verification => false
manage_template => false
ilm_enabled => false
document_type => "logs"
}
stdout {
codec => rubydebug
}
}
I am not sure what’s causing the last error there. Any suggestions how I can fix this?