Security issue while dumping logs to opensearch dashboards

I’m using logstash pipeline to collect and dump logs in Opensearch dashboard. The pipelines.yml file reads like this :

input {
file {
path => “/var/log/opensearch/*”
filter {
json {
source => “message”

output {
opensearch {
hosts => [“localhost:9200”]
index => “logstash-logs-%{+YYYY.MM.dd}”

The pipeline works fine when I remove ‘s’ out of https, and make in opensearch.yml.

It threw following error when I tried with http by commenting the “” portion in opensearch.yml.

[2023-11-09T11:56:09,189][WARN ][logstash.outputs.opensearch][main] Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"http://localhost:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>“OpenSearch Unreachable: [http://localhost:9200/][Manticore::ClientProtocolException] localhost:9200 failed to respond”

If I need a secure connection, what should I do?

@NandanIK When you enable security plugin then you must connect with HTTPS protocol to OpenSearch node.

How did you deploy the cluster?

Is your Logstash running at the same host/vm as OpenSearch nodes?

If your OpenSearch cluster is using self signed certificates then you need to turn off certificate verification or present RootCA of the OpenSearch node in the OpenSearch output’of Logstash config.

Hello @pablo , Thanks for response, yes…!, our 1] Logstash and OpenSearch Running on same Virtual machine, 2] Can you please guide how to turn off self signed certificates,

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production esnode.pem esnode-key.pem root-ca.pem false true esnode.pem esnode-key.pem root-ca.pem true true
#  - CN=kirk,OU=client,O=client,L=test, C=de false internal_opensearch true true ["all_access", "security_rest_api_access"] true [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ######## /etc/opensearch/node1.pem /etc/opensearch/node1-key.pem /etc/opensearch/root-ca.pem true /etc/opensearch/node1.pem /etc/opensearch/node1-key.pem /etc/opensearch/root-ca.pem true
  - 'CN=node1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA' internal_opensearch true true ["all_access", "security_rest_api_access"]

``opensearch.yml file look like this

@Afzal Try the below config with security plugin enabled in OpenSearch.

opensearch {
        hosts => ["https://localhost:9200"]
        # SSL enabled
        ssl => true
        ssl_certificate_verification => false
        # The Logstash Username and Password created earlier
        user => ""
        password => ""
        # The name of the index
        index => "<index_name>"

Hello @pablo , Thanks for the Support, I will try it for sure

Hello @pablo , I need one more help, whenever I tried for pushing data to OpenSearch, I am getting this line for logstash run - "Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>} " Not able to push the logs into opensearch
I request please help me on this,

@Afzal What user did you use?