Issue with CSV file load in OpenSearch with Logstash

Plugins Index Management
1

Hi @all,
We are trying to deploy the logstash for OpenSearch. We are using tarball version.
We are trying to load CSV file in OpenSearch with logstash.
The csv file structure:

c1,c2,c3,c4,c5,c6,c7
2020,H01,99999,"7,33,258",H01,2020,"7,33,258"
2020,H04,99999,"6,60,630",H04,2020,"6,60,630"
2020,H05,99999,"54,342",H05,2020,"54,342"
2020,H07,99999,"18,285",H07,2020,"18,285"
2020,H08,99999,"6,54,872",H08,2020,"6,54,872"
2020,H09,99999,"32,730",H09,2020,"32,730"
2020,H10,99999,"7,509",H10,2020,"7,509"
2020,H11,99999,"26,821",H11,2020,"26,821"
2020,H12,99999,"1,19,387",H12,2020,"1,19,387"
2020,H13,99999,305,H13,2020,305
2020,H14,99999,"1,756",H14,2020,"1,756"
2020,H19,99999,"4,50,394",H19,2020,"4,50,394"
2020,H20,99999,"16,274",H20,2020,"16,274"
2020,H21,99999,"68,109",H21,2020,"68,109"
2020,H22,99999,"69,123",H22,2020,"69,123"
2020,H23,99999,"79,400",H23,2020,"79,400"
2020,H24,99999,"23,96,143",H24,2020,"23,96,143"
2020,H25,99999,"6,65,643",H25,2020,"6,65,643"
2020,H26,99999,"5,62,646",H26,2020,"5,62,646"
2020,H29,99999,"11,67,854",H29,2020,"11,67,854"

There is a total of 37000+ rows in this CSV file.
Here is the configuration file

input {
    file {
        path => "/oss/bin/logstash/test_csv/sample.csv"
        start_position => "beginning"
    }
}

filter {
    csv {
        columns =>[ "c1", "c2", "c3", "c4", "c5", "c6", "c7" ]
    }
    separator => ","
}

output {
    opensearch {
        hosts => “https://reportplus.engrid.in:9200/”
        index => "sample-%{+YYYY.MM.dd}"
        user => "admin"
        password => "admin"
        ssl => true
        cacert => "/oss/bin/opensearch/config/cert1.pem"
    }
}

After executing the logstash file with this config file it gives this error.
command:

 bin/logstash -f config/csv.config

Error:

Using bundled JDK: /oss/bin/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /oss/bin/logstash/logs which is now configured via log4j2.properties
[2022-03-29T17:32:10,074][INFO ][logstash.runner          ] Log4j configuration path used is: /oss/bin/logstash/config/log4j2.properties
[2022-03-29T17:32:10,088][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.16.3", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.13+8 on 11.0.13+8 +indy +jit [linux-x86_64]"}
[2022-03-29T17:32:10,502][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-03-29T17:32:11,562][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-03-29T17:32:12,067][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\" at line 12, column 15 (byte 225) after filter {\n    csv {\n        columns =>[ \"c1\", \"c2\", \"c3\", \"c4\", \"c5\", \"c6\", \"c7\" ]\n    }\n    separator ", :backtrace=>["/oss/bin/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:187:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "/oss/bin/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/oss/bin/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/oss/bin/logstash/logstash-core/lib/logstash/agent.rb:383:in `block in converge_state'"]}
[2022-03-29T17:32:12,189][INFO ][logstash.runner          ] Logstash shut down.
[2022-03-29T17:32:12,200][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
        at oss.bin.logstash.lib.bootstrap.environment.<main>(/oss/bin/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

So if anyone could point me in the right direction that would be a great help.
Thanks.

2

braquets moments

filter {
    csv {
        columns =>[ "c1", "c2", "c3", "c4", "c5", "c6", "c7" ]
        separator => ","
    }
}