Parse already indexed unstructured data to dinamically get structured data ready to be visualized

Hello. Loki + Grafana have a very interesting functionality: from already stored unstructured logs it is possible to create queries by dynamically “structuring” these logs to get the fields that are necessary to make a visualization in place. That is, instead of having taken the precaution of using a DataPrepper’s processor such as “grok” or “key_value” (or similar FluentBit filters, etc.) to store the indexes with the desired structure, it allows you to save the indexes in “raw” form and then , when necessary, “split” its fields on demand to make a specific graph. Is that possible with OpenSearch + Dashboards? Thank you!

You can use script fields functionality to generate new fields from existing fields:

Ah, I didn’t know about it!! Thanks a lot!!!

I fact, this functionality is missing from Opensearch doc. In fact, some page like these should be written there, I think:

I’ll read about it in order to do a “grok”-like painless script applied to my already ingested data. Let’s see if I’m able to do it. Thanks!!

Well, I’ve seen that ingested pipelines could do the job, too:

P.S: More information about scripted fields: Using Painless in Kibana scripted fields | Elastic Blog