Extract severity value from priority field in syslog record

Data Prepper
1

I’m on the latest Opensearch/dataprepper.

I’d like to have a separate field with severity value taken from priority field of the syslog record. I know the calculations:

severity=priority % 8

I could use:

  • add_entries:
    entries:
    - key: “severity”
    value_expression: “/priority % 8”

the problem is dataprepper has no mod (%) math only basic calculations.

Do you have any other ideas here? I even tried Gemini AI, but it was waste of time.

A can’t see any scripting options in dataprepper. Wonder why Gemini suggested script processor with Groove which would do the job.
P.

2

IMHO it is not possible in Dataprepper. I am a bit surprised by that fact. And wonder how to achieve the goal: show me the errors in the syslog in Discover for example. Without this does it make any sense to store syslog records if I cannot trigger alert?

If only DP had more math power.

3

Can you use a modulo equivalent? Never used DP.

You know something like severity = (priority - 8 * (priority / 8))

You’d need something to round down every step, as if they were all integers.

4

I tried this and asked some AI for modulo replacement. To solution needed loops and if/else conditions. So no go for DP. BTW, your expression does not work unfortunately :wink:

5

Hello again!

That’s why I submitted a solution without any loop hehe! I was suspecting you couldn’t use any.

But according to the announcement below, my suggestion is using only basic operators, so it should work? It’s even saying it supports integer type?

Let say priority is 47.

Severity = 47 - 8 * (47 / 8) = 47 - 8 * 5 = 47 - 40 = 7
Severity = 7 same result as modulo.

https://opensearch.org/blog/Announcing-Data-Prepper-2.3.0/#:~:text=The%20add_entries%20processor%20has%20been%20enhanced%20to%20support,value_expression%20option%3A%20-%20add_entries%3A%20entries%3A%20-%20key%3A%20%22request_len%22

6

Almost. The problem with the above calculations is that they must be done with integers but DP uses floats during calculations so as a result I have 0.0 instead of the 7 for example.
And I have no idea if I can force DP to change its behavior. Probably cannot.

There is some hope but only for the future versions, as they reserved ‘%’ character. So some day…

But now I’m forced to move the calculations to fluentbit/lua. Ugly it is, but at least works.
Thanx for your help anyway.