Versions:
OpenSearch v2.11.0
Describe the issue:
I’ve got a self-contained project in which I’m trying to get opensearch to authorize against LDAP (using glauth). I feel like I’m very close but something is failing. I’m pretty sure I have an issue with some of my searches in the authorization_backend section (see below).
I’m hoping someone familiar with the authorization process help me figure out exactly where I’m wrong. The main issue here is the Cannot retrieve roles for User error you’ll see in the output below.
I’m going to be very verbose here (more details are usually better than fewer, right?). Don’t worry about the included creds in this example being posted, they’re only being used in this sandbox and are included here for reproducibility purposes.
background documentation I’ve read
the setup
- 
if you want to get and run this test project: - project saved in this repository
- custom OpenSearch docker image based on opensearchproject/opensearch:2.11.0with a few minor tweaks
- running
- pushd certs && ./generate-certificates.sh ; popd
- set PUIDandPGIDat the top ofdocker-compose.ymlto your current user’s UID/GID
- docker compose pull
- docker compose up -d
- docker compose logs -f
- after a few once the logs have quieted down, ./security-admin-docker-compose.sh
 
 
- 
my entire LDAP database - served by glauth with this config file
- username we’re testing is analyst, password@nalyz3r
- bind DN is cn=binddn,dc=nein,dc=local, passwordr3@d0n!y
 
dn: dc=nein,dc=local
dc: nein
dc: local
objectClass: organizationalUnit
objectClass: dcObject
objectClass: top
dn: ou=groups,dc=nein,dc=local
ou: groups
objectClass: organizationalUnit
objectClass: top
dn: ou=users,dc=nein,dc=local
ou: users
objectClass: organizationalUnit
objectClass: top
dn: ou=Administrator,ou=users,dc=nein,dc=local
ou: Administrator
uid: Administrator
description: Administrator
gidNumber: 500
uniqueMember: cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local
memberUid: analyst
objectClass: posixGroup
objectClass: top
dn: ou=Developers,ou=users,dc=nein,dc=local
ou: Developers
uid: Developers
description: Developers
gidNumber: 501
uniqueMember: cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local
memberUid: analyst
objectClass: posixGroup
objectClass: top
dn: ou=Service,ou=users,dc=nein,dc=local
ou: Service
uid: Service
description: Service
gidNumber: 502
uniqueMember: cn=binddn,ou=Service,ou=users,dc=nein,dc=local
uniqueMember: cn=sensor,ou=Service,ou=users,dc=nein,dc=local
memberUid: binddn
memberUid: sensor
objectClass: posixGroup
objectClass: top
dn: cn=binddn,ou=Service,ou=users,dc=nein,dc=local
cn: binddn
uid: binddn
ou: Service
uidNumber: 5001
accountStatus: active
objectClass: posixAccount
objectClass: shadowAccount
loginShell: /bin/bash
homeDirectory: /home/binddn
description: binddn
gecos: binddn
gidNumber: 502
memberOf: ou=Service,ou=groups,dc=nein,dc=local
shadowExpire: -1
shadowFlag: 134538308
shadowInactive: -1
shadowLastChange: 11000
shadowMax: 99999
shadowMin: -1
shadowWarning: 7
dn: cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local
cn: analyst
uid: analyst
givenName: Analyst
sn: McAnalyzer
ou: Administrator
uidNumber: 1000
accountStatus: active
mail: analyst@nein.local
userPrincipalName: analyst@nein.local
objectClass: posixAccount
objectClass: shadowAccount
loginShell: /bin/bash
homeDirectory: /home/analyst
description: analyst
gecos: analyst
gidNumber: 500
memberOf: ou=Administrator,ou=groups,dc=nein,dc=local
memberOf: ou=Developers,ou=groups,dc=nein,dc=local
shadowExpire: -1
shadowFlag: 134538308
shadowInactive: -1
shadowLastChange: 11000
shadowMax: 99999
shadowMin: -1
shadowWarning: 7
dn: cn=sensor,ou=Service,ou=users,dc=nein,dc=local
cn: sensor
uid: sensor
givenName: Sensor
sn: McSensorface
ou: Service
uidNumber: 1001
accountStatus: active
mail: sensor@nein.local
userPrincipalName: sensor@nein.local
objectClass: posixAccount
objectClass: shadowAccount
loginShell: /bin/bash
homeDirectory: /home/sensor
description: sensor
gecos: sensor
gidNumber: 502
memberOf: ou=Service,ou=groups,dc=nein,dc=local
shadowExpire: -1
shadowFlag: 134538308
shadowInactive: -1
shadowLastChange: 11000
shadowMax: 99999
shadowMin: -1
shadowWarning: 7
- my config.yml
---
_meta:
  type: "config"
  config_version: 2
config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      ldap_auth:
        order: 1
        description: "Authenticate using LDAP"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: ldap
          config:
            enable_ssl: true
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: false
            pemtrustedcas_filepath: /usr/share/opensearch/config/root-ca.pem
            hosts:
            - glauth:636
            bind_dn: cn=binddn,dc=nein,dc=local
            password: "r3@d0n!y"
            userbase: ou=users,dc=nein,dc=local
            usersearch: (uid={0})
            username_attribute: uid
    authz:
      ldap_roles:
        description: "Authorize using LDAP"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: ldap
          config:
            enable_ssl: true
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: false
            pemtrustedcas_filepath: /usr/share/opensearch/config/root-ca.pem
            hosts:
            - glauth:636
            bind_dn: cn=binddn,dc=nein,dc=local
            password: "r3@d0n!y"
            userbase: ou=users,dc=nein,dc=local
            usersearch: (uid={0})
            username_attribute: uid
            rolebase: ou=groups,dc=nein,dc=local
            rolesearch: (memberUid={1})
            userroleattribute: null
            userrolename: none
            rolename: ou
            resolve_nested_roles: true
            skip_users:
              - admin
              - kibanaserver
---
_meta:
  type: "rolesmapping"
  config_version: 2
all_access:
  reserved: false
  backend_roles:
  - "admin"
  - "Administrator"
  description: "Maps admin to all_access"
own_index:
  reserved: false
  users:
  - "*"
  description: "Allow full access to an index named like the username"
kibana_user:
  reserved: false
  backend_roles:
  - "kibanauser"
  - "Developers"
  - "Service"
  description: "Maps kibanauser to kibana_user"
readall:
  reserved: false
  backend_roles:
  - "readall"
  - "Developers"
  - "Service"
manage_snapshots:
  reserved: false
  backend_roles:
  - "snapshotrestore"
  - "Developers"
  - "Service"
kibana_server:
  reserved: true
  users:
  - "kibanaserver"
the process (taken from Docker containers’ logs)
$ curl -u analyst -k https://localhost:9200/_plugins/_security/api/account
- Bind with binddn, successful
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind request basedn=dc=nein,dc=local binddn=cn=binddn,dc=nein,dc=local src=172.27.52.5:41690
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind success binddn=cn=binddn,dc=nein,dc=local src=172.27.52.5:41690
- Search for user analystsuccessful
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Search request basedn=dc=nein,dc=local binddn=cn=binddn,dc=nein,dc=local filter=(uid=analyst) scope=2 searchbasedn=ou=users,dc=nein,dc=local src=172.27.52.5:41690
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Search request special case="top-level users node"
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF AP: Top-Level Users Browse OK filter=(uid=analyst)
- Same search for user analyst, reproduced inldapsearch, successfull:
$ ldapsearch -LLL -H ldap://$(dip opensearch-glauth-1):389 \
    -D 'cn=binddn,dc=nein,dc=local' -w 'r3@d0n!y' \
    -s sub \
    -b 'ou=users,dc=nein,dc=local' \
    '(uid=analyst)'
dn: cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local
cn: analyst
uid: analyst
givenName: Analyst
sn: McAnalyzer
ou: Administrator
uidNumber: 1000
accountStatus: active
mail: analyst@nein.local
userPrincipalName: analyst@nein.local
objectClass: posixAccount
objectClass: shadowAccount
loginShell: /bin/bash
homeDirectory: /home/analyst
description: analyst
gecos: analyst
gidNumber: 500
memberOf: ou=Administrator,ou=groups,dc=nein,dc=local
memberOf: ou=Developers,ou=groups,dc=nein,dc=local
shadowExpire: -1
shadowFlag: 134538308
shadowInactive: -1
shadowLastChange: 11000
shadowMax: 99999
shadowMin: -1
shadowWarning: 7
- Bind with analyst, successful
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind request basedn=dc=nein,dc=local binddn=cn=analyst,ou=administrator,ou=users,dc=nein,dc=local src=172.27.52.5:41696
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind success binddn=cn=analyst,ou=administrator,ou=users,dc=nein,dc=local src=172.27.52.5:41696
- Bind with binddn, successful
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind request basedn=dc=nein,dc=local binddn=cn=binddn,dc=nein,dc=local src=172.27.52.5:41708
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Bind success binddn=cn=binddn,dc=nein,dc=local src=172.27.52.5:41708
- Search for objectClass=*incn=analyst,ou=administrator,ou=users,dc=nein,dc=local, fails?
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Search request basedn=dc=nein,dc=local binddn=cn=binddn,dc=nein,dc=local filter=(objectClass=*) scope=0 searchbasedn=cn=analyst,ou=administrator,ou=users,dc=nein,dc=local src=172.27.52.5:41708
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF Search request default case=
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF AP: Account Search OK filter=(objectClass=*)
opensearch-glauth-1  | Wed, 29 Nov 2023 17:49:20 +0000 INF AP: Search OK filter=(objectClass=*)
opensearch-node1     | [2023-11-29T17:49:20,931][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Cannot retrieve roles for User [name=analyst, backend_roles=[], requestedTenant=null] from ldap due to OpenSearchSecurityException[OpenSearchSecurityException[No user 'cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local' found]]; nested: OpenSearchSecurityException[No user 'cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local' found];
opensearch-node1     | org.opensearch.OpenSearchSecurityException: OpenSearchSecurityException[No user 'cn=analyst,ou=Administrator,ou=users,dc=nein,dc=local' found]
- Same search reproduced with ldapsearch(no results)
$ ldapsearch -LLL -H ldap://$(dip opensearch-glauth-1):389 \
    -D 'cn=binddn,dc=nein,dc=local' -w 'r3@d0n!y' \
    -s base \
    -b 'cn=analyst,ou=administrator,ou=users,dc=nein,dc=local' \
    '(objectClass=*)'