"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n]

Community
1

I was using Opensearch to send logs to Logstash and then output the modified fields to another Opensearch when I stumbled upon this error:

[2025-05-28T06:57:49,669][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>“LogStash::ConfigurationError”, :message=>“Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 6, column 1 (byte 132) after “, :backtrace=>[”/home/onizuwo/logstash-9.0.1/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative’”, “org/logstash/execution/AbstractPipelineExt.java:294:in `initialize’”, “org/logstash/execution/AbstractPipelineExt.java:227:in `initialize’”, “/home/onizuwo/logstash-9.0.1/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize’”, “org/jruby/RubyClass.java:949:in `new’”, “/home/onizuwo/logstash-9.0.1/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute’”, “/home/onizuwo/logstash-9.0.1/logstash-core/lib/logstash/agent.rb:420:in `block in converge_state’”]}

I want to know how to solve it. As the error advised, I tried to provide a comment at line 6 or even leaving a blank space but nothing worked. I also typed the whole conf file by myself without copying and pasting from outside resources as I have seen that this might be the cause of the issue. However, this attempt also didnt work and I am still facing this error and cannot solve it.

Hence, I am posting on the community so that someone can explain to me what is the issue and why is it happening and how to solve. Ultimately, I want to know what I need to do to solve it. Can someone please guide me/advise me on this issue?

2

@Reyhan Please share your Logstash config file.

3 
Code #1
============================================

input {
  opensearch {
    hosts => ["http://opensearch-host:9200"]
    user => "username"
    password => "password"
    index => "wazuh-*"
    ssl_enabled => false
    schedule => "*/5 * * * *"
    docinfo => true
    docinfo_target => "[@metadata][opensearch]"
    size => 1000
  }
}

filter {
  # No filters as this time
}

output {
  stdout {
    codec => rubydebug
  }

Codes #2
============================================
input {
  opensearch {
    hosts => ["https://your-opensearch-node:9200"]
    index => "wazuh-alerts-*"
    user => "your_username"
    password => "your_password"
    ssl => true
    query => '
    {
      "query": {
        "bool": {
          "must": [
            {
              "range": {
                "@timestamp": {
                  "gte": "now-1h",
                  "lte": "now",
                  "format": "strict_date_optional_time"
                }
              }
            },
            {
              "match": {
                "rule.level": {
                  "query": 10,
                  "boost": 1.0
                }
              }
            }
          ]
        }
      },
      "sort": [
        {
          "@timestamp": {
            "order": "asc"
          }
        }
      ]
    }'
    schedule => "* * * * *"
    size => 1000
    scroll => "5m"
    docinfo => true
  }
}
4

@pablo Here’s the Logstash config file above: (Post made by @lsoon3 )

5

@lsoon3 I understand that you’re querying OpenSearch cluster. Did you use OpenSearch input plugin for Logstash?

This is my working test config with OpenSearch input and output plugins.

input {
  opensearch {
    hosts => ["docker1.pablo.local:9200"]
    index => "sample-http-responses"
    query =>  '{ "size":0, "query": { "match_all": {}},"track_total_hits": true }'
    user => "admin"
    password => "Eliatra123"
    ssl  => true
    schedule => "* * * * *"
    docinfo => true
    docinfo_target => 'metadata_with_hash'
   }
}

output{
  opensearch {
      index => "logstash-%{+YYYY.MM.dd}"
      hosts => ["https://node-0.example.com:9200"]
      #user => admin
      #password => Eliatra123
      ssl => true
      ssl_certificate_verification => false
      cacert => "/usr/share/logstash/config/root-ca.pem"
      tls_certificate => "/usr/share/logstash/config/kirk.pem"
      tls_key => "/usr/share/logstash/config/kirk-key.pem"
      action => "create"
  }

}

Does your Logstash fails for both Codes 1 and 2 or just on of them?

6

Yes Logstash fails for both of the codes that we have provided.

7

@Reyhan Do you run these codes separately?

The first code will fail in that shape.

  1. ssl_enabled is an incorrect option
  2. value for hosts is incorrect, this field expects either : or : format

Take a look at my working example again. All options have been tested.

This is your updated configuration. If you’re using docker, be sure that both Logstash and OpenSearch are running in the same network.

input {
  opensearch {
    hosts => ["opensearch-host:9200"]
    user => "username"
    password => "password"
    index => "wazuh-*"
    ssl => true
    schedule => "*/5 * * * *"
    docinfo => true
    docinfo_target => "[@metadata][opensearch]"
    size => 1000
  }
}

filter {
  # No filters as this time
}

output {
  stdout {
    codec => rubydebug
  }
}

In your second code only the host value is incorrect.
Iv’e tested both codes and Logstash didn’t fail.

Are these codes the only ones that are configured in the pipelines.yml ?