Regex supported in Dashboards Query Language (DQL)?

OpenSearch Dashboards
1

Hi

(Reposting this question from slack as we didnt get any feedback there)

Describe the issue:
I’m working on a dashboard visualization, and I need to filter out certain values using a regular expression.
Specifically, I’m trying to match patterns like this:

TEST-[a-z0-9]+/[a-zA-Z0-9-]±ind-core-[a-z0-9]±[a-z][a-z0-9]+

The end goal is to visualize this filtered data as a line graph on the dashboard. However, I’m struggling with how to implement this in the visState JSON section of the dashboard definition.

I can see some examples of using wildcards in the DQL documentation, but in this case, we need to use regex expression (for more complex cases not achievable through wildcards).

Would anyone be able to share the correct syntax or provide any guidance on how I can apply this regex to display the data properly?

Is this not supported at all and should we raise a feature request for this?

Any help or suggestions would be greatly appreciated!

Versions (relevant - OpenSearch/Dashboard): 2.12.0

Thanks,
Shivani

2

Could you provide a specific use case? What is the structure of the documents/logs where you want to perform the filter?

3

I’ll give you an example that works for me with opensearch dashboards 2.15:
–I want to get all documents (logs) that contain a text with this pattern:
“12345678912_123456789123”, that is, two groups of numbers separated by an underscore.
It’s true that with DSL searches it’s not possible, but it is possible with DQL queries. You can add a filter in “opensearch Query DSL” like this:

{
  "query": {
    "regexp": {
      "message": "[0-9]{11}_[0-9]{12}"
    }
  }
}
4

Hi, apologies for the late reply!

The records are in json format, and there is a field in the record, distinguished_name. It contains a hash in the end of the name that either starts with a digit, or an alphabet character.

  • alpha case
    Regex: TEST-[a-z0-9]+/[a-zA-Z0-9-]+/ind-core-[a-z0-9]±[a-z][a-z0-9]+
    Sample string: TEST-vtas251/CON-1/ind-core-677458881-pvplz

  • digit case
    Regex: TEST-[a-z0-9]+/[a-zA-Z0-9-]+/ind-core-[a-z0-9]±[0-9][a-z0-9]+
    Sample string: TEST-vtas251/CON-1/ind-core-677458881-8j6sm

Use-case is to add a filter aggregation to distinguish between the two types.

{
“id”: “3”,
“enabled”: true,
“type”: “filters”,
“params”: {
“filters”: [{
“regexp”: {
“distinguished_name.keyword”: “TEST-[a-z0-9]+/[a-zA-Z0-9-]+/ind-core-[a-z0-9]±[a-z][a-z0-9]+”
},
“label”: “alpha”
}, {
“regexp”: {
“distinguished_name.keyword”: “TEST-[a-z0-9]+/[a-zA-Z0-9-]+/ind-core-[a-z0-9]±[0-9][a-z0-9]+”
},
“label”: “digit”
}
]
},
“schema”: “split”
}

Any thoughts on this?