Getting Missing role error for AD user with all_access role

Security
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 3.6.0 / RHEL 9.7 / Chrome

Describe the issue: We recently built a opensearch cluster for our test environment and we have configured our ADOM group as backend role mapping to all_access role in the cluster. I was able to login to the cluster since last week and all of a sudden, I am not able to get into the cluster. Whenever I am trying to login, authentication is working but authorization is failing and page is immediately redirecting to Missing Role message. I have run _plugins/_security/authinfo with my credentials from POSTMAN and getting empty response for backend_roles. Other members part of different ADOM group with same all_access role are able to login.

2

@Santhosh.Gollapudi Could you share output of the following command. Please hide all sensitive data.

curl --insecure -u admin:<password> "https://localhost:9200/_plugins/_security/api/securityconfig?pretty"
3

Hello @pablo ,

{
  "config": {
    "dynamic": {
      "filtered_alias_mode": "warn",
      "disable_rest_auth": false,
      "disable_intertransport_auth": false,
      "respect_request_indices_options": false,
      "kibana": {
        "multitenancy_enabled": true,
        "private_tenant_enabled": true,
        "default_tenant": "",
        "server_username": "kibanaserver",
        "index": ".kibana",
        "sign_in_options": [
          "BASIC"
        ]
      },
      "http": {
        "anonymous_auth_enabled": false,
        "xff": {
          "enabled": false,
          "internalProxies": "10\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}|192\\.168\\.\\d{1,3}\\.\\d{1,3}|169\\.254\\.\\d{1,3}\\.\\d{1,3}|127\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}|172\\.1[6-9]{1}\\.\\d{1,3}\\.\\d{1,3}|172\\.2[0-9]{1}\\.\\d{1,3}\\.\\d{1,3}|172\\.3[0-1]{1}\\.\\d{1,3}\\.\\d{1,3}",
          "remoteIpHeader": "X-Forwarded-For"
        }
      },
      "authc": {
        "adebp_ldap_auth": {
          "http_enabled": true,
          "order": 3,
          "http_authenticator": {
            "challenge": false,
            "type": "basic",
            "config": {}
          },
          "authentication_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "adebpsmlb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@adebp.vzwcorp.com",
              "password": "***",
              "userbase": "DC=adebp,DC=vzwcorp,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "username_attribute": "sAMAccountName",
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*****ingestion"
              ]
            }
          },
          "description": "ADEBP Active Directory authentication"
        },
        "internal_auth": {
          "http_enabled": true,
          "order": 0,
          "http_authenticator": {
            "challenge": true,
            "type": "basic",
            "config": {}
          },
          "authentication_backend": {
            "type": "intern",
            "config": {}
          },
          "description": "Authenticate via internal users database"
        },
        "uswin_ldap_auth": {
          "http_enabled": true,
          "order": 1,
          "http_authenticator": {
            "challenge": false,
            "type": "basic",
            "config": {}
          },
          "authentication_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "uswinlb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@uswin.ad.vzwcorp.com",
              "password": "***",
              "userbase": "DC=uswin,DC=ad,DC=vzwcorp,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "username_attribute": "sAMAccountName",
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*ingestion"
              ]
            }
          },
          "description": "USWIN Active Directory authentication"
        },
        "vdsi_ldap_auth": {
          "http_enabled": true,
          "order": 2,
          "http_authenticator": {
            "challenge": false,
            "type": "basic",
            "config": {}
          },
          "authentication_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "vdsilb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@vdsi.ent.verizon.com",
              "password": "***",
              "userbase": "DC=vdsi,DC=ent,DC=verizon,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "username_attribute": "sAMAccountName",
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*****ingestion"
              ]
            }
          },
          "description": "VDSI Active Directory authentication"
        }
      },
      "authz": {
        "uswin_ldap_authz": {
          "http_enabled": true,
          "authorization_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "uswinlb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@uswin.ad.vzwcorp.com",
              "password": "***",
              "userbase": "DC=uswin,DC=ad,DC=vzwcorp,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "rolebase": "OU=Groups,DC=uswin,DC=ad,DC=vzwcorp,DC=com",
              "rolesearch": "(member={0})",
              "rolename": "cn",
              "resolve_nested_roles": false,
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*ingestion"
              ]
            }
          },
          "description": "USWIN LDAP group authorisation"
        },
        "adebp_ldap_authz": {
          "http_enabled": true,
          "authorization_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "adebpsmlb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@adebp.vzwcorp.com",
              "password": "***",
              "userbase": "DC=adebp,DC=vzwcorp,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "rolebase": "OU=Groups,DC=adebp,DC=vzwcorp,DC=com",
              "rolesearch": "(member={0})",
              "rolename": "cn",
              "resolve_nested_roles": false,
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*****ingestion"
              ]
            }
          },
          "description": "ADEBP LDAP group authorisation"
        },
        "vdsi_ldap_authz": {
          "http_enabled": true,
          "authorization_backend": {
            "type": "ldap",
            "config": {
              "enable_ssl": true,
              "enable_start_tls": false,
              "enable_ssl_client_auth": false,
              "verify_hostnames": true,
              "pemtrustedcas_filepath": "certs/ldap-ca-bundle.pem",
              "hosts": [
                "vdsilb.vzwcorp.com:636"
              ],
              "follow_referrals": false,
              "bind_dn": "SVC-CSG-ELK-LDAP@vdsi.ent.verizon.com",
              "password": "***",
              "userbase": "DC=vdsi,DC=ent,DC=verizon,DC=com",
              "usersearch": "(sAMAccountName={0})",
              "rolebase": "OU=Groups,DC=vdsi,DC=ent,DC=verizon,DC=com",
              "rolesearch": "(member={0})",
              "rolename": "cn",
              "resolve_nested_roles": false,
              "connect_timeout": 5000,
              "response_timeout": 5000,
              "skip_users": [
                "admin",
                "kibanaserver",
                "kibanaro",
                "logstash",
                "readall",
                "snapshotrestore",
                "csg*ingestion"
              ]
            }
          },
          "description": "VDSI LDAP group authorisation"
        }
      },
      "auth_failure_listeners": {},
      "do_not_fail_on_forbidden": false,
      "multi_rolespan_enabled": true,
      "hosts_resolver_mode": "ip-only",
      "do_not_fail_on_forbidden_empty": false,
      "on_behalf_of": {
        "enabled": false
      }
    }
  }
}
4

@Santhosh.Gollapudi Thank you for sharing the configuration. Is your username unique for each domain? Please be aware that if you use the same username in more than one domain, the first match will be used by the security plugin.

If this is a test environment, can you test your authentication/authorization against a single LDAP domain?

5

I have the same username in VDSI and ADEBP domains and I was able to login with the same configuration until last week. However, I added my id into both of these ADOM groups with all_access role and I am still getting the Missing Role error.

6

@Santhosh.Gollapudi At this point, you don’t know which domain authenticated your user. Since this is a test cluster, you could try to leave only one domain and check which one is failing. Alternatively, you could create a unique user in each domain and see if you get roles from each domain.

7

I will try this. But, I have another user with read only access to opensearch dashboards. He has only one domain for his id and he was able to login earlier but not now.

8

@Santhosh.Gollapudi Which domain out of the shared ones is your read-only user in?
Do you see any errors in OpenSearch logs when that user logs in?

9

Hello, he is part of VDSI domain. It is just showing backend_roles empty:

{“type”:“log”,“@timestamp”:“2026-05-05T13:46:56Z”,“tags”:[“error”,“opensearch”,“data”],“pid”:2785473,“message”:“[security_exception]: no permissions for [indices:data/write/bulk] and User [name=V658321, backend_roles=, requestedTenant=user]”}

10

@Santhosh.Gollapudi I’ve checked and tested your configuration. The only unknown factor for me is the binding user. Can the binding user SVC-CSG-ELK-LDAP@adebp.vzwcorp.com search for groups in the rolebase DN OU=Groups,DC=adebp,DC=vzwcorp,DC=com

Could you test with the ldapsearch tool?

env LDAPTLS_REQCERT=never ldapsearch -b "OU=Groups,DC=adebp,DC=vzwcorp,DC=com" -H ldaps://adebpsmlb.vzwcorp.com:636 -D "SVC-CSG-ELK-LDAP@adebp.vzwcorp.com"

I see that you’ve used the same user for bind_dn in both authc and authz. Is the password the same for both?

11

Hello Pablo,

This has been resolved after adding the filters in the role search as per the thread below:

12

@Santhosh.Gollapudi Thank you for sharing your solution. Just out of curiosity, are workspaces enabled in your OpenSearch Dashboards?

explore.enabled: true
workspace.enabled: true
13

explore is enabled but workspace is disabled. We are using tenants and kept workspaces disabled.