# Opensearch Opensearch-dashboard Import Logfiles

**URL:** <https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717>\
**Category:** OpenSearch\
**Created:** [August 7, 2024, 12:29pm UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717 "2024-08-07T12:29:05Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 7, 2024, 12:29pm UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/1 "2024-08-07T12:29:05Z")

</div>

System: redhat 7,  
installation: On Prem

Hello Community,  
So a few days ago i had to investigate regarding Opensearch because its open source.  
(still using Elasticsearch)  
Now i need help regarding how to import or load logfile/logstash and important part about the security,  
The Documentation on the website is not that good maybe u guys can help me out.

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 8, 2024, 9:23pm UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/2 "2024-08-08T21:23:20Z")

</div>

@amagonawin Could you describe your logs ingest pipeline?  
You can use Logstash with the OpenSearch output plugin or Data Prepper to ingest data to OpenSearch.

How did you deploy your test OpenSearch cluster?

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 9, 2024, 9:01am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/3 "2024-08-09T09:01:52Z")

</div>

Hey, so i have many questions regarding logstash:

first: if i try to use

```auto
bin/logstash-plugin install logstash-output-opensearch 

```

like in the Docu, nothing happens

BUT if i

```auto
bin/logstash -e "input { stdin { } } output { stdout { } }

```

it starts and works… even if i never “installed” it like in the Docu… dont know

I use the logstash plugin.

My pipeline.yml looks like this

```auto
input {
  file {
    path => "/var/log/opensearch/*"
  }
}
filter {
  json {
    source => "message"
 }
}
	
output {
	opensearch {
		hosts => ["//localhost:9200"]
		index => "logstash-log-%(+YYYY.MM.dd)"
	}
}

```

How can i use the logstash at my opensearch dashboard? It doesnt show up?

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 9, 2024, 10:08am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/4 "2024-08-09T10:08:23Z")

</div>

@amagonawin Did you use Logstash OSS with the OpenSearch output plugin.  
The output part of your pipeline.yml is missing few things.  
Check my example below.

```auto
  opensearch {
      index => "logstash-%{+YYYY.MM.dd}"
      hosts => ["https://docker1.pablo.net:9200"]
      user => admin
      password => Eliatra123
      ssl => true
      ssl_certificate_verification => false
      action => "create"
  }

```

If you didn’t disable security plugin or HTTPS on the 9200 endpoint than you need to enable SSL in the OpenSearch output plugin.

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 9, 2024, 10:18am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/5 "2024-08-09T10:18:38Z")

</div>

@pablo  
Thanks for the fast reply  
Hello, i Downloaded : logstash-oss-with-opensearch-output-plugin-8.9.0-linux-x64.tar.gz

Ok, i will give it a shot.

Thanks

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 9, 2024, 10:27am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/6 "2024-08-09T10:27:11Z")

</div>

@pablo  
still dont see any new logfile in my opensearch dashboard.  
i startet it with bin/logstash -e “input { stdin { } } output { stdout { } }”

typed hello world, closed it, but i did not got any log entry…

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 9, 2024, 10:32am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/7 "2024-08-09T10:32:26Z")

</div>

@amagonawin I’ve just downloaded the same binaries and executed the same command.

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/f/f6b6dc1f1743e6335b1a20ac26e28385c5934ae4.jpeg)

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/6/676bb29a6553be8fb077ab3755f1f7e44cca4716.jpeg)

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 9, 2024, 10:43am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/8 "2024-08-09T10:43:54Z")

</div>

@pablo  
that is not the problem

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/1/1f799d2dfb6c41d3b0d868c7493a13e18cff3610.png)

but now i see, somehow he is not taking the changes i made in the yml file…

i also get that but i want to have the log file in my Dashboard, so that i can visuallize it, if you understand what i mean 😃

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 9, 2024, 10:52am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/9 "2024-08-09T10:52:03Z")

</div>

@amagonawin I understood that you didn’t get any output after typing “hello” in the input.  
In that case this is expected. The command line is only for stdin and stdout and it overwrites the pipeline.yml config file. There will be no logs sent to OpenSearch.

To send the logs to OpenSearch you need to use opensearch output as I’ve shared before.  
Also you need to run Logstash using one of the below commands.

```auto
./bin/logstash
./bin/logstash -f /path/to/your/conf.file 

```

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 9, 2024, 11:35am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/10 "2024-08-09T11:35:49Z")

</div>

@pablo  
Thanks

so my conf file was also in conf/logstash.conf

i put the same stuff inside as u showed above.

BUT… still getting errors like:

```auto
[WARN][logstash.outputs.opensearch][main] Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"https://admin:xxxxxx@localhost:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting OpenSearch at URL 'https://localhost:9200/'"}

```

Ok i see, when i created all that i used a " ! " in my password, somehow hes having issues with that, can i somehow change that?

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 9, 2024, 12:55pm UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/11 "2024-08-09T12:55:32Z")

</div>

@amagonawin You can change the password with OpenSearch Dashboards security plugin or with securityadmin.sh and internal\_users.yml file.

If you used demo configuration then logstash:logstash should work.

---

<div class="post-metadata">

**Author:** ![amagonawin](https://avatars.discourse-cdn.com/v4/letter/a/839c29/32.png) [@amagonawin](https://forum.opensearch.org/u/amagonawin)\
**Post date:** [August 12, 2024, 9:43am UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/12 "2024-08-12T09:43:54Z")

</div>

@pablo  
Hey, so its running and all, but now i have another question,

in my dashboard, it is shown, but i dont see the “message” that i tipped in?

just to be clear, if i want to load the logfiles into my dashboard i use:  
 → ./bin/logstash -f /path/to/your/conf.file → do i type the message inside the comand line?

or  
 → bin/logstash -e "input { stdin { } } output { stdout { } } or after i used this command?

This is how it looks like at my dashboards:

 ![image](https://us1.discourse-cdn.com/flex019/uploads/mauve_hedgehog/original/2X/9/94f4353a9905e2bb320f50a779494b9ffbd803c9.png)

Im a bit confused at the moment, thanks

---

<div class="post-metadata">

**Author:** ![pablo](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.opensearch.org/pablo/32/10363_2.png) [@pablo](https://forum.opensearch.org/u/pablo)\
**Post date:** [August 12, 2024, 10:18pm UTC](https://forum.opensearch.org/t/opensearch-opensearch-dashboard-import-logfiles/20717/13 "2024-08-12T22:18:54Z")

</div>

@amagonawin If you want to send stdin{} to OpenSearch then you need a different output.  
This should do the trick.

config.file

```auto
input {
   stdin{}
}
output {
	opensearch {
        index => "logstash-%{+YYYY.MM.dd}"
        hosts => ["https://docker1.pablo.net:9200"]
        user => admin
        password => Eliatra123
        ssl => true
        ssl_certificate_verification => false
        action => "create"
	}
}

```

Also, you can run it as an oneliner.

```auto
./bin/logstash -e 'input { stdin {} } output { opensearch { index => "logstash-%{+YYYY.MM.dd}" hosts => ["https://docker3.pablo.net:9200"] user => "admin" password => "Eliatra123" ssl => true ssl_certificate_verification => false action => "create" } }'

```
