# Logstash is not sending logs to opensearch **URL:** \ **Category:** OpenSearch\ **Created:** [February 8, 2024, 8:05am UTC](https://forum.opensearch.org/t/logstash-is-not-sending-logs-to-opensearch/17810 "2024-02-08T08:05:51Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![harikrishna](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@harikrishna](https://forum.opensearch.org/u/harikrishna)\ **Post date:** [February 8, 2024, 8:05am UTC](https://forum.opensearch.org/t/logstash-is-not-sending-logs-to-opensearch/17810/1 "2024-02-08T08:05:51Z")
**Versions** (relevant - OpenSearch/Dashboard/Server OS/Browser): opensearch-2.6.0-linux-x64.rpm logstash-oss-with-opensearch-output-plugin-8.6.1-linux-x64.tar.gz filebeat-8.6.2-x86\_64.rpm Redis 5.0.3 Red Hat Enterprise Linux release 8.9 (Ootpa) ruby 2.5.9p229 (2021-04-05 revision 67939) [x86\_64-linux] **Describe the issue** : Logstash was ingesting logs to opensearch without any issues for more than a year, but suddenly after OS patching, we are facing the below issue [2024-01-30T12:35:31,071][ERROR][logstash.javapipeline][applogs] Pipeline error {:pipeline\_id=\>“applogs”, :exception=\>#\, :backtrace=\>[“org/jruby/RubyRegexp.java:1145:in `=~'”, “org/jruby/RubyString.java:1684:in `=~'”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:72:in `block in add_patterns_from_file’”, “org/jruby/RubyIO.java:3533:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:70:in `add_patterns_from_file’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:471:in `block in add\_patterns\_from\_files’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:467:in `add\_patterns\_from\_files’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:280:in `block in register’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:276:in `block in register’”, “org/jruby/RubyHash.java:1519:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:271:in `register’”, “org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:234:in `block in register_plugins’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:233:in `register_plugins’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java_pipeline.rb:601:in `maybe\_setup\_out\_plugins’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:246:in `start_workers’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in `run’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:143:in `block in start’”], “pipeline.sources”=\>[“/var/lib/data/logstash/config/pipeline\_app.conf”], :thread=\>“#\”} [2024-01-30T12:35:31,072][INFO][logstash.javapipeline][applogs] Pipeline terminated {“pipeline.id”=\>“applogs”} [2024-01-30T12:35:31,080][INFO][logstash.outputs.opensearch][applogs] Using a default mapping template {:version=\>2, :ecs\_compatibility=\>:v8} [2024-01-30T12:35:31,081][ERROR][logstash.agent] Failed to execute action {:id=\>:applogs, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>“Could not execute action: PipelineAction::Create, action\_result: false”, :backtrace=\>nil} **Configuration** : Logstash pipelines.yml ```auto - pipeline.id: applogs path.config: "/var/lib/data/logstash/config/pipeline_app.conf" - pipeline.id: dblogs path.config: "/var/lib/data/logstash/config/pipeline_db.conf" ``` Logstash pipeline\_app.conf ```auto input { redis { host => "127.0.0.1" port => '6379' data_type => "list" key => "redisqueue" password => "filebeat123" } } filter { if ([fields][logtype] == "UNITS") { grok { tag_on_failure => ["_field_not_found"] match => { "message" => "\s{1,2}(?m)%{DATESTAMP:logtimestamp}%{GREEDYDATA}" } } date { match => ["logtimestamp", "dd.MM.yyyy HH:mm:ss.SSS"] target => "@timestamp" } } else if ([fields][logtype] == "PORTS") { grok { tag_on_failure => ["_field_not_found"] match => { "message" => "\s{1,2}%{DATESTAMP:logtimestamp}%{GREEDYDATA}" } } date { match => ["logtimestamp", "dd.MM.yyyy HH:mm:ss.SSS"] target => "@timestamp" } } else if ([fields][logtype] == "UNITHOST") { grok { tag_on_failure => ["_field_not_found"] match => { "message" => "\s{1,2}(?m)%{DATESTAMP:logtimestamp}%{GREEDYDATA}" } } date { match => ["logtimestamp", "dd.MM.yyyy HH:mm:ss.SSS"] target => "@timestamp" } } if "NORIS" in [tags] { mutate { add_field => { "DataCenter" => "NORIS" } } } if "EIP" in [tags] { mutate { add_field => { "DataCenter" => "EIP" } } } } output { stdout {codec => rubydebug { metadata => true } } if "FIMI_PORT_LOG" in [tags] and "CDE" in [tags] { opensearch { hosts => ["http://OPENSEARCH_LB_IP:PORT"] index => "two-fimiport-logs-%{+YYYY.MM.dd}" user => "admin" password => "PASSWORD" } } if "FIMI_UNIT_LOG" in [tags] and "CDE" in [tags] { opensearch { hosts => ["http://OPENSEARCH_LB_IP:PORT"] index => "two-fimiunit-logs-%{+YYYY.MM.dd}" user => "admin" password => "PASSWORD" } } if "FIMI_UNIT_HOST_LOG" in [tags] and "CDE" in [tags] { opensearch { hosts => ["http://OPENSEARCH_LB_IP:PORT"] index => "two-fimiunithost-logs-%{+YYYY.MM.dd}" user => "admin" password => "PASSWORD" } } } ``` **Relevant Logs or Screenshots** : ---
**Author:** ![harikrishna](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@harikrishna](https://forum.opensearch.org/u/harikrishna)\ **Post date:** [March 18, 2024, 10:18am UTC](https://forum.opensearch.org/t/logstash-is-not-sending-logs-to-opensearch/17810/2 "2024-03-18T10:18:36Z")
> [@harikrishna](#): > > opensearch-2.6.0-linux-x64.rpm Can anyone please tell me what is causing this issue ---
**Author:** ![harikrishna](https://avatars.discourse-cdn.com/v4/letter/h/ecc23a/32.png) [@harikrishna](https://forum.opensearch.org/u/harikrishna)\ **Post date:** [March 18, 2024, 10:32am UTC](https://forum.opensearch.org/t/logstash-is-not-sending-logs-to-opensearch/17810/3 "2024-03-18T10:32:39Z")
[2024-03-18T07:14:38,009][ERROR][logstash.javapipeline][f5logs] Pipeline error {:pipeline\_id=\>“f5logs”, :exception=\>#\, :backtrace=\>[“org/jruby/RubyRegexp.java:1145:in `=~'”, “org/jruby/RubyString.java:1684:in `=~'”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:72:in `block in add_patterns_from_file’”, “org/jruby/RubyIO.java:3533:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:70:in `add_patterns_from_file’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:471:in `block in add\_patterns\_from\_files’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:467:in `add\_patterns\_from\_files’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:280:in `block in register’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:276:in `block in register’”, “org/jruby/RubyHash.java:1519:in `each’”, “/var/lib/data/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-grok-4.4.3/lib/logstash/filters/grok.rb:271:in `register’”, “org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:234:in `block in register_plugins’”, “org/jruby/RubyArray.java:1865:in `each’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:233:in `register_plugins’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java_pipeline.rb:601:in `maybe\_setup\_out\_plugins’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:246:in `start_workers’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in `run’”, “/var/lib/data/logstash/logstash-core/lib/logstash/java\_pipeline.rb:143:in `block in start’”], “pipeline.sources”=\>[“/var/lib/data/logstash/config/pipeline\_f5.conf”], :thread=\>“#\”} [2024-03-18T07:14:38,010][INFO][logstash.javapipeline][f5logs] Pipeline terminated {“pipeline.id”=\>“f5logs”} [2024-03-18T07:14:38,016][ERROR][logstash.agent] Failed to execute action {:id=\>:f5logs, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>“Could not execute action: PipelineAction::Create, action\_result: false”, :backtrace=\>nil} ---
**Author:** ![Gsmitt](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/gsmitt/32/1718_2.png) [@Gsmitt](https://forum.opensearch.org/u/Gsmitt)\ **Post date:** [March 19, 2024, 3:28am UTC](https://forum.opensearch.org/t/logstash-is-not-sending-logs-to-opensearch/17810/4 "2024-03-19T03:28:04Z")
Hey @harikrishna > [@harikrishna](#): > > could not execute action: PipelineAction::Create, action\_result: false > **[Could not execute action: PipelineAction::Create](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create/250802/4)** > > Thank you. Now the Log file say that my custom grok format pattern is not defined. On some hundred lines before I saw that the filter has been adding : Any ideas why? \[2020-10-02T15:35:50,899\]\[DEBUG\]\[logstash.filters.grok \]\[main\] Adding... I’d suggest looking at `/etc/logstash/pipelines.yml` file and check if there are no typos. This is what I see. ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/1/1d1480f28a59962dfabc7ed46a01a06e10e4ae00.png)