Not able to send logs to opensearch from logstash server, getting this warning message in the logs "[o.o.s.a.BackendRegistry ] No 'Authorization' header, send 401 and 'WWW-Authenticate Basic'"

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch 2.7.0

Describe the issue:
Not able to send logs to opensearch from logstash server, getting this warning message in the logs “[o.o.s.a.BackendRegistry ] No ‘Authorization’ header, send 401 and ‘WWW-Authenticate Basic’”

Any help would be highly appreciated

Configuration:

Relevant Logs or Screenshots:

image
image1819×326 39.2 KB

2

Hi @info2kool

Could you send your output in the pipeline.conf file?

3

Hi @Eugene7

Logstash version : 5.6.10
Opensearch version : 2.7.0

After upgrading opensearch version from 1.2.3 to 2.7.0, I am getting these exceptions and logs are sending to opensearch.

Here is the .conf file.

input {
syslog {
type => syslog
port => 51415
}
rabbitmq {
queue => “logs_unicast”
host => [“svrl-dkr1”, “svrl-dkr2”, “svrl-dkr3”]
exchange => “amq.rabbitmq.log”
exchange_type => “topic”
durable => true
key => “#”
type => syslog
add_field => {
“program” => “rabbitmq-log”
}
}
}
filter {
if “docker/” in [program] {
mutate {
add_field => {
“container_id” => “%{program}”
}
}
mutate {
gsub => [
“container_id”, “docker/”, “”
]
}
mutate {
update => [
“program”, “docker”
]
}
}
if “GET / HTTP/1.0” in [message] {
drop{ }
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
user => “admin”
password => “admin”
hosts => [“10.10.10.10:9200”,“10.10.10.10:9200”]
index => “dev-logs-%{+xxxx.ww}”
}
}

4

Have you tried to use the OpenSearch output plugin? If not, please change your output configuration as per the documentation below:

Have you enabled the option plugins.security.ssl.http.enabled ?

5

@Eugene7 I have tried this but opensearch output plugin is not there in that logstash version 5.6.10 which I am using. So it will not recognize this plugin.

Have you enabled the option `plugins.security.ssl.http.enabled? - No I haven’t enabled this ssl security

6

Hi @info2kool
According to the documentation here: Tools - OpenSearch documentation , you use the Logstash version, which is not compatible with your OpenSearch version. Please try to use another version of Logstash.

7

Hi @Eugene7 I installed logstash 7.16.3 version later and still got the same errors. Is it because of the lucene version change in Opensearch 2.7 ? Opensearch 2.7 lucene version is 9 where as logstash lucene versions which I used was 8.

8

@info2kool I found that under output section you use:
elasticsearch {
user => “admin”
password => “admin”
hosts => [“10.10.10.10:9200”,“10.10.10.10:9200”]
index => “dev-logs-%{+xxxx.ww}”
}

Is it still the same after you updated your logstash?
It should be (I might be wrong):
opensearch {
user => “admin”
password => “admin”
hosts => [“10.10.10.10:9200”,“10.10.10.10:9200”]
index => “dev-logs-%{+xxxx.ww}”
}
And in Logstash settings you need to disable ecs.compatibility.mode (don’t remember the actual name of setting)

Moreover in ouput you can specify something like this:
opensearch {
auth_type => {
type => basic
user => “admin”
password => “admin”
}
hosts => [“10.10.10.10:9200”,“10.10.10.10:9200”]
index => “dev-logs-%{+xxxx.ww}”
}