Crashing securityadmin script when export rolesmapping.yml with too many over 250 roles and 8K users each

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

OpenSearch 2.4.0

Describe the issue:

I have created 250 roles through REST API with each having a rolemappings with 8K users. I got the below error. when taking a backup through securityadmin tool.

securityadmin.bat -backup backupfolder -icl -nhnv -cacert …/…/…/config/root-ca.pem -cert …/…/…/config/kirk.pem -key …/…/…/config/kirk-key.pem

Error Message:
FAIL: Get configuration for ‘rolesmapping’ failed because of com.fasterxml.jackson.dataformat.yaml.JacksonYAMLParseException: The incoming YAML document exceeds the limit: 3145728 code points.
at [Source: (ByteArrayInputStream); line: 1, column: 3145723]

Configuration:
Windows distribution with default configurations.

The question is, how can we have a scalable setup for roles and rolemappings. In my case, I have around 90K roles with up to 7K users in each role. How can I satisfy this requirement? Also, what would be the expected range of roles/rolemappings that OpenSearch can handle?

You can reproduce the above with the following python code:

def createRole(name):
url = 'https://localhost:9200/_plugins/_security/api/roles/'+name

body = {
  "cluster_permissions": [
    "cluster_composite_ops",
    "indices_monitor"
  ],
  "index_permissions": [{
    "index_patterns": [
      "movies*"
    ],
    "dls": "",
    "fls": [],
    "masked_fields": [],
    "allowed_actions": [
      "read"
    ]
  }],
  "tenant_permissions": [{
    "tenant_patterns": [
      "human_resources"
    ],
    "allowed_actions": [
      "kibana_all_read"
    ]
  }]
}

res = req.put(url,auth=('admin', 'admin'), verify=False, json = body)
return res.text

def addUsers(roleName, users):
url = 'https://localhost:9200/_plugins/_security/api/rolesmapping/'+roleName

body = {
  "backend_roles" : [  ],
  "hosts" : [ ],
  "users" : users
}

res = req.put(url,auth=('admin', 'admin'), verify=False, json = body)
return res.text

users =

for x in range(8000):
users.append(‘user’+str(x))

for x in range(1,260):
roleName = ‘role_’+str(x)
createRole(roleName)
addUsers(roleName, users)

2

@asfoorial I think this error might be related to the 3 MB limitation of snakeyaml YAML parser used by jackson-dataformats-text.

Have a look at this GitHub issue.

According to the following link, the security plugin uses Jackson-core 2.8.10. However, I’d suggest opening a GitHub issue in OpenSearch security plugin to confirm that and ask for help in solving it.

Please share the link to the GitHub issue once created.