# Open Source Elasticsearch and Kibana **URL:** https://forum.opensearch.org/c/general-elasticsearch/10.md?page=1 [Latest](https://forum.opensearch.org/latest.md) · [Categories](https://forum.opensearch.org/categories.md) · [Tags](https://forum.opensearch.org/tags.md) **Page:** 2 --- ## [Filebeat not pussing every logs to elasticsearch there is discrepancy in the logs](https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485)
**Author:** [@Nandeeshb](https://forum.opensearch.org/u/Nandeeshb)\ **Replies:** 0\ **Last updated:** [July 24, 2024, 6:18am UTC](https://forum.opensearch.org/t/filebeat-not-pussing-every-logs-to-elasticsearch-there-is-discrepancy-in-the-logs/20485 "2024-07-24T06:18:49Z")
Hi Team, Really need a help on this , as Im not able to figure out whats the exact issue. Filebeat version: 8.3.3 Elastic search version installed on vms: 8.12.0 We are using filebeat to push the logs to elastic sear… --- ## [ReIndexing index data from ElasticSearch 7.14.0 to OpenSearch 2.14.0 using ReIndex API](https://forum.opensearch.org/t/reindexing-index-data-from-elasticsearch-7-14-0-to-opensearch-2-14-0-using-reindex-api/20407)
**Author:** [@nadiw98](https://forum.opensearch.org/u/nadiw98)\ **Replies:** 7\ **Last updated:** [July 23, 2024, 2:38am UTC](https://forum.opensearch.org/t/reindexing-index-data-from-elasticsearch-7-14-0-to-opensearch-2-14-0-using-reindex-api/20407 "2024-07-23T02:38:19Z")
I’m trying to reindex data from ES to OS through reindex API. When indexing data to the ES, previously we had set custom routing shards allocations for the documents. When reindexing the data to OS, I want to keep the sa… --- ## [Issue with Nested Sorting in Opensearch Query](https://forum.opensearch.org/t/issue-with-nested-sorting-in-opensearch-query/20338)
**Author:** [@b.kakulte](https://forum.opensearch.org/u/b.kakulte)\ **Replies:** 1\ **Last updated:** [July 17, 2024, 12:48pm UTC](https://forum.opensearch.org/t/issue-with-nested-sorting-in-opensearch-query/20338 "2024-07-17T12:48:04Z")
Hi everyone, I’m encountering an issue with sorting nested fields in an Elasticsearch query. Despite following the documentation, the sorting doesn’t seem to work as expected. I’m hoping someone can point out what I’m d… --- ## [Discover page and read data](https://forum.opensearch.org/t/discover-page-and-read-data/20073)
**Author:** [@m\_pahlevanzadeh](https://forum.opensearch.org/u/m_pahlevanzadeh)\ **Replies:** 2\ **Last updated:** [July 1, 2024, 11:02am UTC](https://forum.opensearch.org/t/discover-page-and-read-data/20073 "2024-07-01T11:02:09Z")
Discover page read data correctly. with a date and time correctly. But I can’t read correctly with SQL. How can I read data such as Discover page? --- ## [Runtime fields on opensearch](https://forum.opensearch.org/t/runtime-fields-on-opensearch/9837)
**Author:** [@devarpisheth](https://forum.opensearch.org/u/devarpisheth)\ **Replies:** 2\ **Last updated:** [June 28, 2024, 2:54pm UTC](https://forum.opensearch.org/t/runtime-fields-on-opensearch/9837 "2024-06-28T14:54:00Z")
We need to run painless scripts on main document on opensearch. It’s part of 7.17.4 Runtime field allows that from elastic --- ## [OpenSearch plugin in Logstash (8.13.4) unavailable](https://forum.opensearch.org/t/opensearch-plugin-in-logstash-8-13-4-unavailable/19681)
**Author:** [@tomse](https://forum.opensearch.org/u/tomse)\ **Replies:** 1\ **Last updated:** [June 24, 2024, 10:30am UTC](https://forum.opensearch.org/t/opensearch-plugin-in-logstash-8-13-4-unavailable/19681 "2024-06-24T10:30:25Z")
I’m about to setup an ELK or should I say OLOD? stack using latest (2.12) opensearch and opensearch-dashboards and logstash 8.13.4 I can see the logstash (from docker hub) doesn’t contain the opensearch plugin. all of… --- ## [Automating Rule Creation for Kibana](https://forum.opensearch.org/t/automating-rule-creation-for-kibana/19904)
**Author:** [@emin](https://forum.opensearch.org/u/emin)\ **Replies:** 0\ **Last updated:** [June 19, 2024, 3:43pm UTC](https://forum.opensearch.org/t/automating-rule-creation-for-kibana/19904 "2024-06-19T15:43:40Z")
I use curl to create the rule: curl -k -X POST "https://192.168.10.131:5601/api/detection\_engine/rules/\_bulk\_action" -d"{"rule\_id":"process\_started\_by\_ms\_office\_program\_possible\_payload","risk\_score":50,"description":"Pr… --- ## [Intermittent Connection Timeouts and "No Living Connections" Issues with AWS OpenSearch](https://forum.opensearch.org/t/intermittent-connection-timeouts-and-no-living-connections-issues-with-aws-opensearch/19803)
**Author:** [@Ganapathi](https://forum.opensearch.org/u/Ganapathi)\ **Replies:** 0\ **Last updated:** [June 12, 2024, 9:13am UTC](https://forum.opensearch.org/t/intermittent-connection-timeouts-and-no-living-connections-issues-with-aws-opensearch/19803 "2024-06-12T09:13:46Z")
Hi everyone, We have hosted our Elasticsearch using AWS OpenSearch service with the following configuration: Instance type: m5.large.search EBS volume size: 150 GiB Provisioned IOPS: 3000 IOPS Provisioned Throughput: … --- ## [KQL, Lucenc, MS KQL, Doctrine QL, ESQL and so on](https://forum.opensearch.org/t/kql-lucenc-ms-kql-doctrine-ql-esql-and-so-on/19665)
**Author:** [@m\_pahlevanzadeh](https://forum.opensearch.org/u/m_pahlevanzadeh)\ **Replies:** 0\ **Last updated:** [June 1, 2024, 9:33am UTC](https://forum.opensearch.org/t/kql-lucenc-ms-kql-doctrine-ql-esql-and-so-on/19665 "2024-06-01T09:33:55Z")
I’m confused, I need to use advanced query in discover or write filter for visualization, But KQL doesn’t have ref, and has weak tutorial. For example I need to pipe but I can’t use them in discover. Please guide me to… --- ## [Delete duplicated data in query result](https://forum.opensearch.org/t/delete-duplicated-data-in-query-result/19663)
**Author:** [@m\_pahlevanzadeh](https://forum.opensearch.org/u/m_pahlevanzadeh)\ **Replies:** 0\ **Last updated:** [June 1, 2024, 6:07am UTC](https://forum.opensearch.org/t/delete-duplicated-data-in-query-result/19663 "2024-06-01T06:07:03Z")
Suppose I have field service in result, it return HTTPS, PING‌ and many.But more than 1000 HTTPS or more than 200 PING , I only want to see one for each. before any help, thank you. --- ## [Can you run multiple instance of opensource on 1 physical server](https://forum.opensearch.org/t/can-you-run-multiple-instance-of-opensource-on-1-physical-server/19447)
**Author:** [@jojamali](https://forum.opensearch.org/u/jojamali)\ **Replies:** 0\ **Last updated:** [May 19, 2024, 12:37am UTC](https://forum.opensearch.org/t/can-you-run-multiple-instance-of-opensource-on-1-physical-server/19447 "2024-05-19T00:37:07Z")
I am trying to get multiple instances of elasticsearch working on 1 server to no avail and can’t get any help. I was wondering if it was possible with this. I am working on an Arkime build and have large but limited ser… --- ## [Can we send data from open search to kafka using logstash](https://forum.opensearch.org/t/can-we-send-data-from-open-search-to-kafka-using-logstash/19406)
**Author:** [@ram-narasimha](https://forum.opensearch.org/u/ram-narasimha)\ **Replies:** 0\ **Last updated:** [May 16, 2024, 5:53am UTC](https://forum.opensearch.org/t/can-we-send-data-from-open-search-to-kafka-using-logstash/19406 "2024-05-16T05:53:19Z")
I want to send some filtered logs from open search dashboard to kafka topics. is it possible to do that --- ## [Take input fro user or keyboard like service name or application name and retrieve data from elasticsearch](https://forum.opensearch.org/t/take-input-fro-user-or-keyboard-like-service-name-or-application-name-and-retrieve-data-from-elasticsearch/19004)
**Author:** [@nnn](https://forum.opensearch.org/u/nnn)\ **Replies:** 0\ **Last updated:** [April 23, 2024, 4:30pm UTC](https://forum.opensearch.org/t/take-input-fro-user-or-keyboard-like-service-name-or-application-name-and-retrieve-data-from-elasticsearch/19004 "2024-04-23T16:30:26Z")
Hello, I have written below code to accept input from keyboard and retrieve data from elasticsearch cluster from elasticsearch import Elasticsearch from datetime import datetime es = Elasticsearch (“https://localhost”… --- ## [Kibana doesn't redirect to login screen after session timeout until I interact](https://forum.opensearch.org/t/kibana-doesnt-redirect-to-login-screen-after-session-timeout-until-i-interact/18940)
**Author:** [@satyamvehere](https://forum.opensearch.org/u/satyamvehere)\ **Replies:** 0\ **Last updated:** [April 19, 2024, 10:29am UTC](https://forum.opensearch.org/t/kibana-doesnt-redirect-to-login-screen-after-session-timeout-until-i-interact/18940 "2024-04-19T10:29:37Z")
I want kinbana to redirect to login page when session times out! opendistro\_security.cookie.ttl: 1800000 opendistro\_security.session.ttl: 60000 i have kept these, so after 60 secs kibana’s session should time out and … --- ## [Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target](https://forum.opensearch.org/t/caused-by-sun-security-validator-validatorexception-pkix-path-building-failed-sun-security-provider-certpath-suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/7734)
**Author:** [@sharath](https://forum.opensearch.org/u/sharath)\ **Replies:** 13\ **Last updated:** [April 16, 2024, 7:16pm UTC](https://forum.opensearch.org/t/caused-by-sun-security-validator-validatorexception-pkix-path-building-failed-sun-security-provider-certpath-suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/7734 "2024-04-16T19:16:01Z")
We getting below error in Kubernetes Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requ… --- ## [Charts only using 5 colors](https://forum.opensearch.org/t/charts-only-using-5-colors/18658)
**Author:** [@CarolynR](https://forum.opensearch.org/u/CarolynR)\ **Replies:** 1\ **Last updated:** [April 14, 2024, 10:49pm UTC](https://forum.opensearch.org/t/charts-only-using-5-colors/18658 "2024-04-14T22:49:20Z")
When I create a line or pie chart, they only use 5 colors. If there are more than 5 data series (more than 5 pie slices), the 5 colors are reused. Is there to get opensearch to use more than 5 colors by default? I am … --- ## [ISSue for adding agent](https://forum.opensearch.org/t/issue-for-adding-agent/18828)
**Author:** [@Aryan](https://forum.opensearch.org/u/Aryan)\ **Replies:** 1\ **Last updated:** [April 12, 2024, 12:01am UTC](https://forum.opensearch.org/t/issue-for-adding-agent/18828 "2024-04-12T00:01:02Z")
Hy, everyone i hope all of you very well, i have issue during for agent adding time i’m not able to add agent i have installed wazuh through quickstart on my ubuntu virtual machine when i want to add agent to my window… --- ## [Logstash config for Multiple pipelines usage](https://forum.opensearch.org/t/logstash-config-for-multiple-pipelines-usage/18623)
**Author:** [@gray653](https://forum.opensearch.org/u/gray653)\ **Replies:** 6\ **Last updated:** [April 3, 2024, 8:29am UTC](https://forum.opensearch.org/t/logstash-config-for-multiple-pipelines-usage/18623 "2024-04-03T08:29:02Z")
Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): OpenSearch 2.12 Logstash 8.9.0 Ubuntu 20.04 Firefox Describe the issue: Thing is i am trying to set up a Logstash pipeline with Auditbeat and Filebeat a… --- ## [Imported index pattern missing fields](https://forum.opensearch.org/t/imported-index-pattern-missing-fields/18662)
**Author:** [@CarolynR](https://forum.opensearch.org/u/CarolynR)\ **Replies:** 0\ **Last updated:** [April 1, 2024, 5:10pm UTC](https://forum.opensearch.org/t/imported-index-pattern-missing-fields/18662 "2024-04-01T17:10:59Z")
My scenario is that I export some index patterns from one instance of OpenSearch and import them into another instance of OpenSearch. This can be for migration or moving from development to production. The new instance… --- ## [Migrating from ElasticSearch 7.16.1 to OpenSearch](https://forum.opensearch.org/t/migrating-from-elasticsearch-7-16-1-to-opensearch/18035)
**Author:** [@Arveen](https://forum.opensearch.org/u/Arveen)\ **Replies:** 5\ **Last updated:** [March 27, 2024, 10:40am UTC](https://forum.opensearch.org/t/migrating-from-elasticsearch-7-16-1-to-opensearch/18035 "2024-03-27T10:40:19Z")
Hi, I have an Elasticsearch cluster with 3 nodes, each running version 7.16.1, and Kibana version 7.4.2 on each node. Our company has decided to migrate to OpenSearch. After researching the migration process, I found th… --- ## [How to increase log storage duration](https://forum.opensearch.org/t/how-to-increase-log-storage-duration/18470)
**Author:** [@arshayubm](https://forum.opensearch.org/u/arshayubm)\ **Replies:** 3\ **Last updated:** [March 26, 2024, 9:15am UTC](https://forum.opensearch.org/t/how-to-increase-log-storage-duration/18470 "2024-03-26T09:15:58Z")
How can we increase log storage duration of domains, For Example lets say I want to keep log storage for 7 days, how can I achieve it --- ## [Logstash pipeline with Auditbeat and Filebeat](https://forum.opensearch.org/t/logstash-pipeline-with-auditbeat-and-filebeat/18513)
**Author:** [@gray653](https://forum.opensearch.org/u/gray653)\ **Replies:** 5\ **Last updated:** [March 26, 2024, 1:23am UTC](https://forum.opensearch.org/t/logstash-pipeline-with-auditbeat-and-filebeat/18513 "2024-03-26T01:23:19Z")
Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): OpenSearch 2.12 Logstash 8.9.0 Ubuntu 20.04 Firefox Describe the issue: I have installed and managed to use Opensearch and Opensearch Dashboard, as well… --- ## [Trying to read logs through opentelemetry and data prepper but facing an error](https://forum.opensearch.org/t/trying-to-read-logs-through-opentelemetry-and-data-prepper-but-facing-an-error/18248)
**Author:** [@rjain1328](https://forum.opensearch.org/u/rjain1328)\ **Replies:** 3\ **Last updated:** [March 21, 2024, 10:43pm UTC](https://forum.opensearch.org/t/trying-to-read-logs-through-opentelemetry-and-data-prepper-but-facing-an-error/18248 "2024-03-21T22:43:39Z")
Hi Team… Can anyone help me out ? For POC i have setup (Opensearch, Kafka,Opentelemetry,Data prepper). in one server. My All services are running fine but when i am doing curl command to opensearch to get the index d… --- ## [Getting Log Information in the Notification of an alert](https://forum.opensearch.org/t/getting-log-information-in-the-notification-of-an-alert/18500)
**Author:** [@Gamer44](https://forum.opensearch.org/u/Gamer44)\ **Replies:** 1\ **Last updated:** [March 21, 2024, 12:41pm UTC](https://forum.opensearch.org/t/getting-log-information-in-the-notification-of-an-alert/18500 "2024-03-21T12:41:43Z")
I have set up an alert with a webhook to Microsoft Teams. This is working fine so far. However, I want to include relevant information from the log, that triggered the alert, in the message sent to Teams. This informatio… --- ## [It is possible to explode a string in opensearch?](https://forum.opensearch.org/t/it-is-possible-to-explode-a-string-in-opensearch/11498)
**Author:** [@olaf](https://forum.opensearch.org/u/olaf)\ **Replies:** 2\ **Last updated:** [February 14, 2024, 7:46pm UTC](https://forum.opensearch.org/t/it-is-possible-to-explode-a-string-in-opensearch/11498 "2024-02-14T19:46:10Z")
Hello there! New to Opensearch here. I was simply wondering if it is possible to explode a string in opensearch the same way I could do in SQL for instance… The classic example being a document with a variable containi… --- ## [How can I change the height of the arc shape in Vega?](https://forum.opensearch.org/t/how-can-i-change-the-height-of-the-arc-shape-in-vega/17895)
**Author:** [@Aigerim](https://forum.opensearch.org/u/Aigerim)\ **Replies:** 0\ **Last updated:** [February 14, 2024, 7:35am UTC](https://forum.opensearch.org/t/how-can-i-change-the-height-of-the-arc-shape-in-vega/17895 "2024-02-14T07:35:32Z")
I am currently developing a visualization using Vega version 5 for OpenSearch. My data and reference materials are sourced from the Vega project on GitHub. The editor I use: https://vega.github.io/ . I want to adjust t… --- ## [Has anyone come across this while executing yarn osd bootstrap?](https://forum.opensearch.org/t/has-anyone-come-across-this-while-executing-yarn-osd-bootstrap/9126)
**Author:** [@vhlam.oci](https://forum.opensearch.org/u/vhlam.oci)\ **Replies:** 2\ **Last updated:** [January 25, 2024, 8:14am UTC](https://forum.opensearch.org/t/has-anyone-come-across-this-while-executing-yarn-osd-bootstrap/9126 "2024-01-25T08:14:30Z")
@mapbox/hast-util-table-cell-style@0.2.0: The engine “node” is incompatible with this module. Expected version “\>=12”. Got “10.24.1” --- ## [Can we upload plain text files and JFR also in the OpenSearch by any means?](https://forum.opensearch.org/t/can-we-upload-plain-text-files-and-jfr-also-in-the-opensearch-by-any-means/17465)
**Author:** [@amitks77](https://forum.opensearch.org/u/amitks77)\ **Replies:** 1\ **Last updated:** [January 19, 2024, 11:18am UTC](https://forum.opensearch.org/t/can-we-upload-plain-text-files-and-jfr-also-in-the-opensearch-by-any-means/17465 "2024-01-19T11:18:33Z")
We are sharing our software stats and bug reports for other teams with the centralized opensearch. All the logs and alarms are picked up by opensearch preety smmothly. But the delta remains are : Few command outputs F… --- ## [ES 6.8.6 upgrade to OpenSearch 2.x](https://forum.opensearch.org/t/es-6-8-6-upgrade-to-opensearch-2-x/17210)
**Author:** [@mpreddy](https://forum.opensearch.org/u/mpreddy)\ **Replies:** 0\ **Last updated:** [December 21, 2023, 7:38am UTC](https://forum.opensearch.org/t/es-6-8-6-upgrade-to-opensearch-2-x/17210 "2023-12-21T07:38:06Z")
Hi Team, We upgraded from ES 6.8.6 to OS 1.2.3. Now we want to upgrade to OpenSearch 2.x. Any indices which are created in ES 6.8.6 should be re-indexed in OS 1.2.3 before we upgrade to OpenSearch 2.x which we are handli… --- ## [\[ConnectionError\]: connect ECONNREFUSED 172.29.0.2:9200" (Opensearch + Dashboards + Docker)](https://forum.opensearch.org/t/connectionerror-connect-econnrefused-172-29-0-2-9200-opensearch-dashboards-docker/10547)
**Author:** [@scull7](https://forum.opensearch.org/u/scull7)\ **Replies:** 1\ **Last updated:** [December 20, 2023, 4:35pm UTC](https://forum.opensearch.org/t/connectionerror-connect-econnrefused-172-29-0-2-9200-opensearch-dashboards-docker/10547 "2023-12-20T16:35:43Z")
Hi All, I’m attempting to set up a working opensearch + dashboards on inside docker containers on a DigitalOcean virtual machine (running ubuntu 20.04). With the below docker-compose.yml file the containers load proper… [Previous page](https://forum.opensearch.org/c/general-elasticsearch/10.md) [Next page](https://forum.opensearch.org/c/general-elasticsearch/10.md?page=2)