Regex that applied field masking, which works in OpenSearch 2x but not in OpenSearch 3x

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): OpenSearch 3.3.0

Describe the issue: We recently moved from Opensearch 2.19.0 to Opensearch 3.3.0.
For certain fields, we are using partial field masking. According to this forum discussion: Pattern-based field masking disables the search , we realize that a role with masked_fields cannot query on masked fields since it is purposefully prohibited by the 3pp. However, the logs were visible in the Opensearch 2.19.0 version.
After we just upgraded to Opensearch 3.3.0, we saw that the role with masked_fields could no longer see logs at all, not only query.

Checking the logs, we found this error:
“org.opensearch.security.privileges.PrivilegesConfigurationValidationException: A regular expression needs to be wrapped in /…”

Then, we updated the regex by wrapping it like:- /pattern/ , now it’s working.

old one:-
masked_fields:

  • “message::(\[(admin)?priv\d*\].?\[/(admin)?priv\d\])::*"
  • "extra_data.::(\[(admin)?priv\d*\].?\[/(admin)?priv\d*\])::*”

new one:-
masked_fields:

  • “message::/(\[(admin)?priv\d*\].?\[/(admin)?priv\d\])/::*"
  • "extra_data.::/(\[(admin)?priv\d*\].?\[/(admin)?priv\d*\])/::*”

We want to know if the OS upgrade included stronger regex validation in masked_fields.

Configuration:

old one:-
masked_fields:

  • “message::(\[(admin)?priv\d*\].?\[/(admin)?priv\d\])::*"
  • "extra_data.::(\[(admin)?priv\d*\].?\[/(admin)?priv\d*\])::*”

new one:-
masked_fields:

  • “message::/(\[(admin)?priv\d*\].?\[/(admin)?priv\d\])/::*"
  • "extra_data.::/(\[(admin)?priv\d*\].?\[/(admin)?priv\d*\])/::*”

Relevant Logs or Screenshots:

“org.opensearch.security.privileges.PrivilegesConfigurationValidationException: A regular expression needs to be wrapped in /…”

2

@chirumanem According to OpenSearch documentation, the notation <field>::/<regular-expression>/::<replacement-string> was always required. It could be that it wasn’t enforced. OpenSearch documentation has followed this format since 1.x

3

Hi @pablo ,
Thank you for responding and clarifying. Then how this worked until the 2x version, but not simply with the new upgrade. This strict regex inspection applies only to the new uplift, not to the beginning.

Regards,

Chiranjeevi