Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
OpenSearch/Dashboard
Describe the issue:
Would it be possible to provide support for natural language query in the Discover page please? i.e. in addition to being able to ask a query in DQL or Lucene, a natural language query could be made instead.
Apologies if this is something that has been requested elsewhere or already included in future plans - any input re that would also be really valuable, thank you.
Reasoning:
What would be valuable is a means to ask a query about ingested logs in natural language, e.g. ‘what errors are in my logs?’ and then the Discover page updates the histogram and search results in the same way as is currently performed when asking a DQL or Lucene based query.
Support for natural language queries is possible using the Opensearch Dashboard Assistant and the queries are converted to PPL (I believe, based on documentation). However the results to queries within the assistant chat window are shorter responses e.g. summaries of the log entries and do not update the search results shown in Discover.
Some challenges from brief investigation into this:
1: Does the existing natural language query to PPL conversion rely on vector search, or otherwise how does/would the conversion/translation have knowledge of valid field names in the index pattern/ingested log to then know which field to query? I don’t have enough knowledge of how the existing implementation for the Dashboard assistant plugin currently does this.
2: I did spend some time investigating if a plugin could be implemented to add a new language in the Discover page but found that the Discover page itself is currently a plugin, so a) any extension of this would mean duplication of the existing code and b) there are a number of posts indicating that the Discover page will be reworked (so any extension work performed now would have to be redone at some point in the future), see Reinvent Discover Plugin · Issue #1792 · opensearch-project/OpenSearch-Dashboards · GitHub, [Feature] Unified query language selection in Data Explorer · Issue #5504 · opensearch-project/OpenSearch-Dashboards · GitHub
Configuration:
Currently testing a stack using a demo configuration of Opensearch Dashboards v2.19, Opensearcg v2.19 with logs being ingested into index patterns using logstash 9.0.3 and filebeat 9.0.3.
Relevant Logs or Screenshots:
N/A