Server OS: Ubuntu 22.04, running in Docker containers
Browser: Firefox
Describe the issue:
Hello, OpenSearch Community,
I’m reaching out to explore the capabilities of OpenSearch Dashboards in terms of log visualization and analysis. Specifically, I’m interested in understanding if there’s an existing functionality or a recommended approach to visually combine multiple log documents into a single, unified view within the dashboard.
The typical use case involves analyzing logs from various sources, where each log is a separate document. Instead of viewing these logs as individual entries, I’m looking for a way to aggregate them visually. This could mean combining logs by specific criteria (e.g., time frame, event type, or any other common field) to create a more cohesive and comprehensive overview of the data.
Here are a few points I’m particularly interested in:
Existing Features: Are there built-in features or plugins within OpenSearch Dashboards that support this type of document aggregation for visualization purposes?
Custom Solutions: If there’s no out-of-the-box functionality, has anyone developed a custom solution or workaround that achieves a similar result?
Best Practices: Are there recommended practices or tools within the OpenSearch ecosystem that facilitate this kind of data presentation?
I’m open to exploring creative solutions, including custom visualizations, scripts, or any integrations that might help achieve this unified view of logs. Any guidance, suggestions, or examples from your experiences would be greatly appreciated.
Thank you in advance for your insights and support.
So what you are attempting to do is “aggregate” something in your logs. In order to do this you need to have the logs parsed into smaller fields, or running a search and using the number of logs matching over time (for example) to provide a metric essentially. There are a lot of good use cases for doing this including alerting, dashboarding, reporting. OpenSearch supports all of these use cases and things. Can you describe what you are trying to solve instead of just stating what you are trying to do with the tool?
Look at the image. If I wanna see the full log message I need to click on the lens. I would like to have a view where you can view each log without timestamp. Just the log message without any metadata and timestamp etc. Just the log messages. Like the following:
125.68.54.153 - - [2018-08-11T11:32:33.526Z] GET /styles/main.css HTTP/1.1 200 3116 - Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1
197.250.60.222 - - [2018-08-11T11:29:54.281Z] GET /opensearch/opensearch-1.0.0.deb HTTP/1.1 200 5374 - Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1
88.83.239.95 - - [2018-08-11T11:25:46.292Z] GET /styles/ad-blocker.css HTTP/1.1 200 6036 - Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.50 Safari/534.24
251.234.26.249 - - [2018-08-11T11:24:39.013Z] GET /enterprise HTTP/1.1 200 6446 - Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.50 Safari/534.24
230.58.7.204 - - [2018-08-11T11:20:41.025Z] GET /beats/filebeat/filebeat-6.3.2-linux-x86_64.tar.gz HTTP/1.1 200 9914 - Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1
Just like the way logs are represented in the terminal.
What I wanna solve is, that I dont need to click on each lense to see the full log message but rather have a view where can I view the full log message for each document like you would see in a terminal
It seems like an entirely different question than the first one, I would suggest a new thread due to the new question. If you would rather not use the web UI for log analytics you can always interact with OpenSearch via CLI or API. ex : SQL and PPL CLI - OpenSearch Documentation
