Opensearch ldaps

Using opensearch 1.3.2 in a docker environment on redhat and trying to configure ldaps.
But with a twist: i’d like to test this out without having certificates for my nodes/dashboards. As our environment only has secure ldap servers, i’m wondering if this is possible as it’s a hassle getting the required certificates.

some more info:

my config.yml

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: false
        transport_enabled: false
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: internal
      ldap_auth:
        order: 1
        description: "Authenticate using LDAP"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: ldap
          config:
            enable_ssl: true
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: false
            hosts:
            - xxxxxxxx:636 
            bind_dn: 'CN=xxxx,OU=xxxx,OU=xxxx,DC=xxx,DC=xxxxx'
            password: 'xxxxx'
            userbase: 'OU=x,OU=x,OU=x,OU=x,DC=x,DC=x'
            usersearch: '(mail={0})'
            username_attribute: null

my opensearch_dashboards.yml (<== is this correct or does this need to be opensearch.yml?)

server.port: 5601
server.host: "0.0.0.0"
opensearch.hosts: ["http://localhost:9200"]
opensearch.username: kibanaserver
opensearch.password: kibanaserver
server.ssl.enabled: false
opensearch.ssl.verificationMode: none
opensearch_security.multitenancy.enabled: false
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
# Use this setting if you are running kibana without https
opensearch_security.cookie.secure: false

my docker-compose.yml:

version: '3.5'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:1.3.2
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.type=single-node
      - bootstrap.memory_lock=false # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    configs:
      - source: ldap_auth_elk
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml
      - source: opensearch_roles
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml
    volumes:
      - esdata:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:1.3.2
    container_name: opensearch-dashboards
    environment:
      - OPENSEARCH_HOSTS=["https://opensearch-node1:9200"]
    ports:
      - 5601:5601
    configs:
      - source: opensearch_kibana
        target: /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml    
    networks:
      - opensearch-net
  logstash:
    image: opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.3
    container_name: logstash
    configs:
     - source: opensearch_logstash
       target: /config-dir/logstash_http_json.conf
    command: logstash -f /config-dir/logstash_http_json.conf
    volumes:
    - logstash:/config-dir
    environment:
      - OPENSEARCH_HOSTS='["https://opensearch-node1:9200"]'
    ports:
      - 5043:5043
    networks:
      - opensearch-net
volumes:
  esdata:
  logstash:
configs:
  ldap_auth_elk:
    external: true
  opensearch_kibana:
    external: true
  opensearch_roles:
    external: true
  opensearch_logstash:
    external: true
networks:
  opensearch-net:

@Scarecrow Yes, it is possible.
Your config is not fully HTTP as you still refer to HTTPS in opensearch-dashboards and logstash.

You need to disable SSL on HTTP and comment out/remove the remaining plugins.security.http in opensearch.yml as per the below example.

plugins.security.ssl.http.enabled: false
#plugins.security.ssl.http.pemcert_filepath: esnode.pem
#plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
#plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem

Then set OPENSEARCH_HOSTS=["http://opensearch-node1:9200"] in opensearch-dashboards and logstash section of docker-compose.yml file.

In regards to config.yml, if your DC uses a self-signed certificate then you must add it to the config. The DC certificate file has to be mapped (through volumes) or copied(through Dockerfile) to /usr/share/opensearch/config/ in the OpenSearch container.

        config:
            enable_ssl: true
            pemtrustedcas_filepath: "name of the DC cert file"

I understand the first part (setting the traffic between dashboard and log stash in http instead of https) but not sure how it’s related to having secure ldap connection between my organisation’s ldap server and the dashboard.

I’m forced to have secure ldap connection, but as far as I understand I need certificates in my opensearch dashboards config, which I’d rather avoid. I’m sorry if I misunderstood

@Scarecrow To be clear, you have to keep the secure connection between OpenSearch and LDAP, but you don’t want a secure connection between OpenSearch and OpenSearch Dashboards.

Am I correct?

@pablo only the connection between open search and ldap needs to be secure, the rest is “optional” if it’s secure good, if not also ok.

@Scarecrow OpenSearch Dashboards doesn’t make any connection with LDAP, it’s done by OpenSearch.

Therefore, OpenSearch Dashboards can connect to OpenSearch using HTTPS without configuring any certificates and with server.ssl.enabled set to false.

Your opensearch_dashboards.yml file is correct.

My suggestion was to turn off SSL on OpenSearch if you’d like to use only HTTP between OpenSearch - OpenSearch Dashboards, and OpenSearch - Logstash.

no worries, the https between opensearch - dashboards and logstash is all working, the only thing that is not working is the LDAPS configuration.

i’m getting an error in the logs:

["error","plugins","securityDashboards"],"pid":1,"message":"Failed authentication: Error: Authentication Exception"}

and 

"res":{"statusCode":401,"responseTime":148,"contentLength":9},"message":"POST /auth/login 401 148ms - 9.0B"}

@Scarecrow Could you share the full config.yml? I’m missing authz section.

Have you tested your binding with DC?

Dooes your DC use self-signed certificate?

@Scarecrow Could you also check for errors in OpenSearch logs?

config.yml: I thought only the auth section was enough to get it working, especially in combination with role mapping?
DC does not use a self signed certificate.

For the logs on the node:

[2022-06-14T12:18:14,375][WARN ][c.a.d.a.l.b.LDAPAuthorizationBackend] [opensearch-node1] Unable to connect to ldapserver xxxx:636 due to OpenSearchException[Empty file path for plugins.security.ssl.transport.truststore_filepath]. Try next.
[2022-06-14T12:18:14,376][WARN ][o.o.s.a.BackendRegistry  ] [opensearch-node1] Authentication finally failed for xxxxxxxxx from 10.0.30.7:51804```

@Scarecrow auth section does only authentication. If you want to base your authorization on AD usernames only and OpenSearch internal groups then authz is not needed.

However, you could use AD groups as backend roles in roles_mapping.yml. Then you would need authz section to pull the groups from AD.

According to your error, OpenSearch can’t find the certificate trustore file.

Empty file path for plugins.security.ssl.transport.truststore_filepath

Could you share your opensearch.yml file?

I caved and let my ICT department create a p7b file :blush:

I’m using portainer to deploy my stack, so I’ve created a config which contains the content of my p7b, then made it available to my opensearch node:

version: '3.5'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:1.3.2
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.type=single-node
      - bootstrap.memory_lock=false # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    configs:
      - source: ldap_auth_elk
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml
      - source: opensearch_roles
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml
      - source: cert.pem
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/cert.pem
    volumes:
      - esdata:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:1.3.2
    container_name: opensearch-dashboards
    environment:
      - OPENSEARCH_HOSTS=["https://opensearch-node1:9200"]
    ports:
      - 5601:5601
    configs:
      - source: opensearch_kibana
        target: /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml    
    networks:
      - opensearch-net
  logstash:
    image: opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.3
    container_name: logstash
    configs:
     - source: opensearch_logstash
       target: /config-dir/logstash_http_json.conf
    command: logstash -f /config-dir/logstash_http_json.conf
    volumes:
    - logstash:/config-dir
    environment:
      - OPENSEARCH_HOSTS='["https://opensearch-node1:9200"]'
    ports:
      - 5043:5043
    networks:
      - opensearch-net
volumes:
  esdata:
  logstash:
configs:
  ldap_auth_elk:
    external: true
  opensearch_kibana:
    external: true
  opensearch_roles:
    external: true
  opensearch_logstash:
    external: true
  cert.pem: 
    external: true

networks:
  opensearch-net:

Then i’ve added the trustpath in my ldap_auth_elk config (which is my config.yml):

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: false
        transport_enabled: false
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: internal
      ldap_auth:
        order: 1
        description: "Authenticate using LDAP"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: ldap
          config:
            enable_ssl: true
            enable_start_tls: false
            enable_ssl_client_auth: false
            pemtrustedcas_filepath: /usr/share/opensearch/plugins/opensearch-security/securityconfig/cert.pem
            verify_hostnames: false
            hosts:
            - xxxxxx:636
            bind_dn: 'CN=xxxxx,OU=xxx,OU=xxxxx,DC=xxxx,DC=xxxx'
            password: 'xxxxx'
            userbase: 'OU=xxx,OU=xxxx,OU=xxx,OU=xxxx,DC=xxxxx,DC=xxxxx'
            usersearch: '(mail={0})'
            username_attribute: null
			
    authz:
      ldap_roles:
        description: "Authorize using LDAP"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: ldap
          config:
            enable_ssl: true
            pemtrustedcas_filepath: /usr/share/opensearch/plugins/opensearch-security/securityconfig/cert.pem
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            hosts:
            -xxxxxxx:636
            bind_dn: 'CN=xxxx,OU=xxx,OU=xxxx,DC=xxxx,DC=xxxxxx'
            password: 'xxxxxxx'
            userbase: 'OU=xxxx,OU=xxxxx,OU=xxxxx,OU=xxxxx,DC=xxx,DC=xxxxx'
            usersearch: '(mail={0})'
            username_attribute: cn
            skip_users:
              - admin
              - kibanaserver
            rolebase: 'OU=xxxx,OU=xxxx,OU=xxxx,OU=xxxx,DC=xxxx,DC=xxxx'
            rolesearch: memberOf
            userroleattribute: null
            userrolename: disabled
            rolename: cn
            resolve_nested_roles: true

Now i’m getting the following error:

[2022-06-14T14:41:25,670][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)

However from what I see in the documentation, I need both a key and a pem to run securityadmin, or am I missing something?

the opensearch_dashboard.yml you were asking:

server.port: 5601
server.host: "0.0.0.0"
opensearch.hosts: ["http://localhost:9200"]
opensearch.username: kibanaserver
opensearch.password: kibanaserver
server.ssl.enabled: false
opensearch.ssl.verificationMode: none
opensearch_security.multitenancy.enabled: false
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.multitenancy.enable_filter: false
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
# Use this setting if you are running kibana without https
opensearch_security.cookie.secure: false

thanks a heaps already!

@Scarecrow I was looking for opensearch.yml rather than opensearch_dashboards.yml.
You’ve already shared opensearch_dashboards.yml in your first post.

@Scarecrow In regards to securityadmin.sh, you need both cert and key.
The securityadmin.sh requires admin certs.

Did you apply your config to the cluster using securityadmin.sh?

In regards to the cert.pem file. Could you move it to /usr/share/opensearch/config and use the below line in authc and authz?

pemtrustedcas_filepath: "cert.pem"

I don’t have (or have not adapted) the opensearch.yml, guess that is part of my problem :wink:

I’ll look into moving the cert.pem to the /usr/share/opensearch/config location.

I got to say i’m a bit confused: I thought the securityadmin.sh is normally for the configuration of https intra cluster, and not for communication towards the ldap?

update on the different changes you’ve requested:

docker-compose:

version: '3.5'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:1.3.2
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.type=single-node
      - bootstrap.memory_lock=false # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    configs:
      - source: ldap_auth_elk
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml
      - source: opensearch_roles
        target: /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml
      - source: cert.pem
        target: /usr/share/opensearch/config/cert.pem
    volumes:
      - esdata:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:1.3.2
    container_name: opensearch-dashboards
    environment:
      - OPENSEARCH_HOSTS=["https://opensearch-node1:9200"]
    ports:
      - 5601:5601
    configs:
      - source: opensearch_kibana
        target: /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml    
    networks:
      - opensearch-net
  logstash:
    image: opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.3
    container_name: logstash
    configs:
     - source: opensearch_logstash
       target: /config-dir/logstash_http_json.conf
    command: logstash -f /config-dir/logstash_http_json.conf
    volumes:
    - logstash:/config-dir
    environment:
      - OPENSEARCH_HOSTS='["https://opensearch-node1:9200"]'
    ports:
      - 5043:5043
    networks:
      - opensearch-net
volumes:
  esdata:
  logstash:
configs:
  ldap_auth_elk:
    external: true
  opensearch_kibana:
    external: true
  opensearch_roles:
    external: true
  opensearch_logstash:
    external: true
  cert.pem: 
    external: true

networks:
  opensearch-net:

config.yml

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: false
        transport_enabled: false
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: internal
      ldap_auth:
        order: 1
        description: "Authenticate using LDAP"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: ldap
          config:
            enable_ssl: true
            pemtrustedcas_filepath: "cert.pem"
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: false
            hosts:
            - xxxxxxxxx:636
            bind_dn: 'CN=xxxx,OU=xxx,OU=xx,DC=x,DC=x'
            password: 'xxxxxxx'
            userbase: 'OU=xxx,OU=x,OU=x,OU=x,DC=x,DC=xxxx'
            usersearch: '(mail={0})'
            username_attribute: null
			
    authz:
      ldap_roles:
        description: "Authorize using LDAP"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: ldap
          config:
            enable_ssl: true
            pemtrustedcas_filepath: "cert.pem"
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            hosts:
            - xxxxxx:636
            bind_dn: 'CN=xxxxx,OU=x,OU=x,DC=xxx,DC=xxx'
            password: 'xxxx'
            userbase: 'OU=x,OU=x,OU=x,OU=x,DC=x,DC=x'
            usersearch: '(mail={0})'
            username_attribute: cn
            skip_users:
              - admin
              - kibanaserver
            rolebase: 'OU=x,OU=x,OU=x,OU=x,DC=x,DC=x'
            rolesearch: memberOf
            userroleattribute: null
            userrolename: disabled
            rolename: cn
            resolve_nested_roles: true

Still complains about the securityadmin.sh. Not sure how I’m going to obtain a key and cert (as you’ve might have guessed i’m not home in the certificate world, i’m just messing around in the edges.)

@Scarecrow I’ve tested your config and I’ve found that extra line above the authz section contains tab key

           username_attribute: null
			
    authz:

This wasn’t reported during the docker-compose startup, but when I’ve tried to backup the existing config with securityadmin.sh it complained about this tab key. Please be sure that no tab keys is in the file config files. Use only space to manage indents of the yaml file.

Could you try to remove that empty line in your config and test docker-compose again?

           username_attribute: null		
    authz:

The mentioned certificates are in the /usr/share/opensearch/config of your OpenSearch docker.

When you use docker-compose and map the security config files, then you don’t need to run securityadmin.sh script as the config is applied when docker-compose builds the environment.

If you’d like to change the config of the running container and apply it then you need to use the securityadmin.sh script.

The below commands should be executed inside the container.

Backup config

"/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -backup "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv

Apply config

"/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv

@Scarecrow Just noticed in your config.yml. Your basic authentication is disabled.
You’ll need it if you’d like to use OpenSearch Dashboards. OpenSearch Dashboards uses the service account kibanaserver to access OpenSearch.

Set http_enabled: true to allow basic authentication.

alrighty :smiley: we’re getting somewhere

Some stuff I had found out myself already (like the TAB error) but the http_enabled to true was a headscratcher since I just got an error “kibanaserver is unable to authenticate”.
I’m guessing thats the trouble with changing the configs that much as i have…

I’m able to log in now via LDAP authentication, so thats working, now it’s just messing around with the roles/authz parts so it recognizes me as admin.

A HUGE thanks already, and i’ll report back here with my final configs if I finish so it might help other people :slight_smile:

alrighty running into something strange with my roles:
from the roles_mapping.yml

all_access:
  reserved: false
  backend_roles:
  - "admin"
  - "my_ad_group"   #the group my user belongs to
  description: "Maps admin to all_access"

When I log in via LDAP the security tab is not visible, and when i go to “index management” I get the following error:

[security_exception] no permissions for [cluster:admin/opendistro/ism/policy/search] and User [name=CN=xxxx

When I log in via de admin/admin creds I can see my_ad_group as a backend role to the role “all_access”
Strange