Logstash input plugin s3snssqs issue

Hi Team,

I have been tetsing logstash s3snssqs plugin to fetch logs from sqs s3.

Earlier the stack was s3 ← sqs ← filebeat → logstash → opensearch

Now we are trying to eliminate filebeat and want to grab logs through the input plugin
It worked fine in dev env ( test ) and now I am trying to implement the same in qa env but while configuring the s3snssqs plugin as input we are receiving a log count is 100’s ( filebeat service was stopped at this point ) where as with filebeat config we are receiving as expected logs ( which is in min 1k logs per minute )

Anyone faced this type of issue with this plugin ( s3snssqs ) or is it something happening in the backend between sqs and the plugin trying to fetch the logs.

Here’s the sample snippet of the logstash config input section

  s3snssqs {
    region                     => "us-west-1"
    from_sns                   => false
    consumer_threads           => 2
    s3_default_options         => { "endpoint_discovery" => true }
    queue                      => "n11-us-va-03-alb-logs-message-queue"
    access_key_id              => "#########"
    secret_access_key          => "#############"
    type                       => "sqs-logs"
    tags                       => ["alb-logs"]
    sqs_skip_delete            => false
    codec                      => json
    s3_options_by_bucket       => [
      { bucket_name => "n11-us-va-03-alb-logs-message-queue"
        folders => [
          { key => ".*/alb.*"
            codec => "json_stream"
            type => "ALB"}

Can anyone please help me with this please, Thank you

There was a issue with Logstash configuration which was reading only first line of the json file in s3
Thanks to this below link. this resolved my issue.