LDAP integration with OPENSEARCH

Security
1

Hello

I am new to opensearch, please help me with this issue, I am trying to integrate ldap with opensource community Opensearch, but I am not able to login to the Opensearch dashboard.
This is config.yml

`---

# This is the main OpenSearch Security configuration file where authentication
# and authorization is defined.
#
# You need to configure at least one authentication domain in the authc of this file.
# An authentication domain is responsible for extracting the user credentials from
# the request and for validating them against an authentication backend like Active Directory for example.
#
# If more than one authentication domain is configured the first one which succeeds wins.
# If all authentication domains fail then the request is unauthenticated.
# In this case an exception is thrown and/or the HTTP status is set to 401.
#
# After authentication authorization (authz) will be applied. There can be zero or more authorizers which collect
# the roles from a given backend for the authenticated user.
#
# Both, authc and auth can be enabled/disabled separately for REST and TRANSPORT layer. Default is true for both.
#        http_enabled: true
#        transport_enabled: true
#
# For HTTP it is possible to allow anonymous authentication. If that is the case then the HTTP authenticators try to
# find user credentials in the HTTP request. If credentials are found then the user gets regularly authenticated.
# If none can be found the user will be authenticated as an "anonymous" user. This user has always the username "anonymous"
# and one role named "anonymous_backendrole".
# If you enable anonymous authentication all HTTP authenticators will not challenge.
#
#
# Note: If you define more than one HTTP authenticators make sure to put non-challenging authenticators like "proxy" or "clientcert"
# first and the challenging one last.
# Because it's not possible to challenge a client with two different authentication methods (for example
# Kerberos and Basic) only one can have the challenge flag set to true. You can cope with this situation
# by using pre-authentication, e.g. sending a HTTP Basic authentication header in the request.
#
# Default value of the challenge flag is true.
#
#
# HTTP
#   basic (challenging)
#   proxy (not challenging, needs xff)
#   kerberos (challenging)
#   clientcert (not challenging, needs https)
#   jwt (not challenging)
#   host (not challenging) #DEPRECATED, will be removed in a future version.
#                          host based authentication is configurable in roles_mapping

# Authc
#   internal
#   noop
#   ldap

# Authz
#   ldap
#   noop



_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    # Set filtered_alias_mode to 'nowarn' to allow more than 2 filtered aliases per index silently
    #filtered_alias_mode: warn
    #do_not_fail_on_forbidden: false
    #kibana:
    # Kibana multitenancy
    #multitenancy_enabled: true
    #private_tenant_enabled: true
    #default_tenant: ""
    #server_username: kibanaserver
    #index: '.kibana'
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
        internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        #internalProxies: '.*' # trust all internal proxies, regex pattern
        #remoteIpHeader:  'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
      kerberos_auth_domain:
        http_enabled: false
        transport_enabled: false
        order: 6
        http_authenticator:
          type: kerberos
          challenge: true
          config:
            # If true a lot of kerberos/security related debugging output will be logged to standard out
            krb_debug: false
            # If true then the realm will be stripped from the user name
            strip_realm_from_principal: true
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 4
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
      proxy_auth_domain:
        description: "Authenticate via proxy"
        http_enabled: false
        transport_enabled: false
        order: 3
        http_authenticator:
          type: proxy
          challenge: false
          config:
            user_header: "x-proxy-user"
            roles_header: "x-proxy-roles"
        authentication_backend:
          type: noop
      jwt_auth_domain:
        description: "Authenticate via Json Web Token"
        http_enabled: false
        transport_enabled: false
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: "base64 encoded HMAC key or public RSA/ECDSA pem key"
            jwt_header: "Authorization"
            jwt_url_parameter: null
            jwt_clock_skew_tolerance_seconds: 30
            roles_key: null
            subject_key: null
        authentication_backend:
          type: noop
      clientcert_auth_domain:
        description: "Authenticate via SSL client certificates"
        http_enabled: false
        transport_enabled: false
        order: 2
        http_authenticator:
          type: clientcert
          config:
            username_attribute: cn #optional, if omitted DN becomes username
          challenge: false
        authentication_backend:
          type: noop
      ldap:
        description: "Authenticate via LDAP or Active Directory"
        http_enabled: true
        transport_enabled: true
        order: 5
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          # LDAP authentication backend (authenticate users against a LDAP or Active Directory)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
              - ldaps://vm0u*********.corp.chartercom.com
            bind_dn: cn=svc-vds-specenteradd,ou=users,ou=serviceaccts,ou=security,dc=corp,dc=chartercom,dc=com
            password: ********
            userbase: 'ou=users,ou=authentication,ou=security,dc=corp,dc=chartercom,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(sAMAccountName={0})'
            # Use this attribute from the user as username (if not set then DN is used)
            username_attribute: cn
    authz:
      roles_from_myldap:
        description: "Authorize via LDAP or Active Directory"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          # LDAP authorization backend (gather roles from a LDAP or Active Directory, you have to configure the above LDAP authentication backend settings too)
          type: ldap
          config:
            # enable ldaps
            enable_ssl: false
            # enable start tls, enable_ssl should be false
            enable_start_tls: false
            # send client certificate
            enable_ssl_client_auth: false
            # verify ldap hostname
            verify_hostnames: true
            hosts:
              - ldaps://vm0**********.corp.chartercom.com
            bind_dn: cn=svc-vds-specenteradd,ou=users,ou=serviceaccts,ou=security,dc=corp,dc=chartercom,dc=com
            password: **********
            userbase: 'ou=users,ou=authentication,ou=security,dc=corp,dc=chartercom,dc=com'
            usersearch: '(sAMAccountName={0})'
            username_attribute: cn
            rolebase: 'ou=authentication,ou=security,dc=corp,dc=chartercom,dc=com'
            # Filter to search for roles (currently in the whole subtree beneath rolebase)
            # {0} is substituted with the DN of the user
            # {1} is substituted with the username
            # {2} is substituted with an attribute value from user's directory entry, of the authenticated user. Use userroleattribute to specify the name of the attribute
            rolesearch: '(member={0})'
            # Specify the name of the attribute which value should be substituted with {2} above
            userroleattribute: null
            # Roles as an attribute of the user entry
            userrolename: disabled
            #userrolename: memberOf
            # The attribute in a role entry containing the name of that role, Default is "name".
            # Can also be "dn" to use the full DN as rolename.
            rolename: cn
            # Resolve nested roles transitive (roles which are members of other roles and so on ...)
            resolve_nested_roles: true
            userbase: 'ou=users,ou=authentication,ou=security,dc=corp,dc=chartercom,dc=com'
            # Filter to search for users (currently in the whole subtree beneath userbase)
            # {0} is substituted with the username
            usersearch: '(uid={0})'
            # Skip users matching a user name, a wildcard or a regex pattern
            #skip_users:
            #  - 'cn=Michael Jackson,ou*people,o=TEST'
            #  - '/\S*/'
      roles_from_another_ldap:
        description: "Authorize via another Active Directory"
        http_enabled: false
        transport_enabled: false
        authorization_backend:
          type: ldap
          #config goes here ...
  #    auth_failure_listeners:
  #      ip_rate_limiting:
  #        type: ip
  #        allowed_tries: 10
  #        time_window_seconds: 3600
  #        block_expiry_seconds: 600
  #        max_blocked_clients: 100000
  #        max_tracked_clients: 100000
  #      internal_authentication_backend_limiting:
  #        type: username
  #        authentication_backend: intern
  #        allowed_tries: 10
  #        time_window_seconds: 3600
  #        block_expiry_seconds: 600
  #        max_blocked_clients: 100000
  #        max_tracked_clients: 100000
`

This is my opensearch.yml

# ======================== OpenSearch Configuration =========================
#
# NOTE: OpenSearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.opensearch.org
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: VM-UAT
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: vm0unelkdla0001.corp.chartercom.com
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /apps/opensearch/data
#
# Path to log files:
#
path.logs: /apps/opensearch/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: vm0unelkdla0001.corp.chartercom.com
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["22.242.73.84"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
#
cluster.initial_cluster_manager_nodes: ["22.242.73.84"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the optimization of datetime formatters caching along with change in default datetime formatter
# Once there is no observed impact on performance, this feature flag can be removed.
#
#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false
#
# Gates the functionality of enabling Opensearch to use pluggable caches with respective store names via setting.
#
opensearch.experimental.feature.pluggable.caching.enabled: false
plugins.security.ssl.transport.pemcert_filepath: certificates/opensearch-certificate.pem
plugins.security.ssl.transport.pemkey_filepath: certificates/key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/opensearch-certificate.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: certificates/opensearch-certificate.pem
plugins.security.ssl.http.pemkey_filepath: certificates/key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/opensearch-certificate.pem
#plugins.security.ssl.transport.pemcert_filepath: esnode.pem
#plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
#plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
#plugins.security.ssl.transport.enforce_hostname_verification: false
#plugins.security.ssl.http.enabled: false
#plugins.security.ssl.http.pemcert_filepath: esnode.pem
#plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
#plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
#plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
#plugins.security.authcz.admin_dn: ['']
plugins.security.audit.type: internal_opensearch
#plugins.security.enable_snapshot_restore_privilege: true]
#plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector, .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task, .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta, .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config, .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*, .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*, .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources, .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models, .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates, .plugins-flow-framework-state]
#node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
#
logger.org.opensearch.index.reindex: debug
plugins.security.nodes_dn_dynamic_config_enabled: true

I am not getting this error & warning

 yellow cluster state....
[2024-04-24T07:29:48,018][INFO ][o.o.s.s.ConfigHelper     ] [vm0unelkdla0001.corp.chartercom.com] Will update 'config' with /etc/opensearch/opensearch-security/config.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2024-04-24T07:29:48,022][ERROR][o.o.s.c.ConfigurationRepository] [vm0unelkdla0001.corp.chartercom.com] Cannot apply default config (this is maybe not an error!)
java.security.PrivilegedActionException: null
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:575) ~[?:?]
        at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:86) ~[opensearch-security-2.13.0.0.jar:2.13.0.0]
        at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:65) ~[opensearch-security-2.13.0.0.jar:2.13.0.0]
        at org.opensearch.security.configuration.ConfigurationRepository.initalizeClusterConfiguration(ConfigurationRepository.java:158) [opensearch-security-2.13.0.0.jar:2.13.0.0]
        at org.opensearch.security.configuration.ConfigurationRepository.lambda$initOnNodeStart$0(ConfigurationRepository.java:318) [opensearch-security-2.13.0.0.jar:2.13.0.0]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: com.fasterxml.jackson.core.JsonParseException: Duplicate field 'userbase'
 at [Source: (FileReader); line: 223, column: 21]
        at com.fasterxml.jackson.core.json.JsonReadContext._checkDup(JsonReadContext.java:250) ~[jackson-core-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.core.json.JsonReadContext.setCurrentName(JsonReadContext.java:244) ~[jackson-core-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:497) ~[jackson-dataformat-yaml-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.core.JsonParser.nextFieldName(JsonParser.java:1140) ~[jackson-core-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:539) ~[jackson-databind-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100) ~[jackson-databind-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25) ~[jackson-databind-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4947) ~[jackson-databind-2.17.0.jar:2.17.0]
        at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3269) ~[jackson-databind-2.17.0.jar:2.17.0]
        at org.opensearch.security.support.ConfigHelper.fromYamlReader(ConfigHelper.java:166) ~[opensearch-security-2.13.0.0.jar:2.13.0.0]
        at org.opensearch.security.support.ConfigHelper.fromYamlFile(ConfigHelper.java:181) ~[opensearch-security-2.13.0.0.jar:2.13.0.0]
        at org.opensearch.security.support.ConfigHelper.lambda$uploadFile$0(ConfigHelper.java:88) ~[opensearch-security-2.13.0.0.jar:2.13.0.0]
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:571) ~[?:?]
        ... 5 more
[2024-04-24T07:29:48,172][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing on REST API is enabled.
[2024-04-24T07:29:48,173][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from REST API auditing.
[2024-04-24T07:29:48,173][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing on Transport API is enabled.
[2024-04-24T07:29:48,173][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from Transport API auditing.
[2024-04-24T07:29:48,173][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing of request body is enabled.
[2024-04-24T07:29:48,173][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Bulk requests resolution is disabled during request auditing.
[2024-04-24T07:29:48,174][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Index resolution is enabled during request auditing.
[2024-04-24T07:29:48,174][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Sensitive headers auditing is enabled.
[2024-04-24T07:29:48,174][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing requests from kibanaserver users is disabled.
[2024-04-24T07:29:48,174][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing request headers <NONE> is disabled.
[2024-04-24T07:29:48,174][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing request url params <NONE> is disabled.
[2024-04-24T07:29:48,176][WARN ][o.o.s.a.r.AuditMessageRouter] [vm0unelkdla0001.corp.chartercom.com] No endpoint configured for categories [BAD_HEADERS, FAILED_LOGIN, MISSING_PRIVILEGES, GRANTED_PRIVILEGES, OPENDISTRO_SECURITY_INDEX_ATTEMPT, SSL_EXCEPTION, AUTHENTICATED, INDEX_EVENT, COMPLIANCE_DOC_READ, COMPLIANCE_DOC_WRITE, COMPLIANCE_EXTERNAL_CONFIG, COMPLIANCE_INTERNAL_CONFIG_READ, COMPLIANCE_INTERNAL_CONFIG_WRITE], using default endpoint
[2024-04-24T07:29:48,176][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing of external configuration is disabled.
[2024-04-24T07:29:48,176][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing of internal configuration is enabled.
[2024-04-24T07:29:48,176][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing only metadata information for read request is enabled.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing will watch {} for read requests.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing read operation requests from kibanaserver users is disabled.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing only metadata information for write request is enabled.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing diffs for write requests is disabled.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing write operation requests from kibanaserver users is disabled.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Auditing will watch <NONE> for write requests.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] .opendistro_security is used as internal security index.
[2024-04-24T07:29:48,177][INFO ][o.o.s.a.i.AuditLogImpl   ] [vm0unelkdla0001.corp.chartercom.com] Internal index used for posting audit logs is null
[2024-04-24T07:29:48,178][INFO ][o.o.s.c.ConfigurationRepository] [vm0unelkdla0001.corp.chartercom.com] Hot-reloading of audit configuration is enabled
[2024-04-24T07:29:48,178][INFO ][o.o.s.c.ConfigurationRepository] [vm0unelkdla0001.corp.chartercom.com] Node 'vm0unelkdla0001.corp.chartercom.com' initialized
[2024-04-24T07:29:48,190][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Indexing [429] fieldMappingDocs
[2024-04-24T07:29:48,190][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Indexing [429] fieldMappingDocs
[2024-04-24T07:29:48,652][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Loaded [429] field mapping docs successfully!
[2024-04-24T07:29:48,665][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Loaded [429] field mapping docs successfully!
[2024-04-24T07:29:48,674][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Indexing [23] customLogTypes
[2024-04-24T07:29:48,681][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Indexing [23] customLogTypes
[2024-04-24T07:29:48,731][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Loaded [23] customLogType docs successfully!
[2024-04-24T07:29:48,736][INFO ][o.o.s.l.LogTypeService   ] [vm0unelkdla0001.corp.chartercom.com] Loaded [23] customLogType docs successfully!
[2024-04-24T07:29:48,790][INFO ][o.o.s.i.DetectorIndexManagementService] [vm0unelkdla0001.corp.chartercom.com] info deleteOldIndices
[2024-04-24T07:29:48,790][INFO ][o.o.s.i.DetectorIndexManagementService] [vm0unelkdla0001.corp.chartercom.com] info deleteOldIndices
[2024-04-24T07:29:48,791][INFO ][o.o.s.i.DetectorIndexManagementService] [vm0unelkdla0001.corp.chartercom.com] No Old Finding Indices to delete
[2024-04-24T07:29:48,791][INFO ][o.o.s.i.DetectorIndexManagementService] [vm0unelkdla0001.corp.chartercom.com] No Old Alert Indices to delete
[2024-04-24T07:29:56,985][INFO ][o.o.m.a.MLModelAutoReDeployer] [vm0unelkdla0001.corp.chartercom.com] Index not found, not performing auto reloading!
[2024-04-24T07:29:56,986][INFO ][o.o.m.c.MLCommonsClusterManagerEventListener] [vm0unelkdla0001.corp.chartercom.com] Starting ML sync up job...
[2024-04-24T07:30:00,379][INFO ][o.o.p.PluginsService     ] [vm0unelkdla0001.corp.chartercom.com] PluginService:onIndexModule index:[security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg]
[2024-04-24T07:30:00,383][INFO ][o.o.c.m.MetadataCreateIndexService] [vm0unelkdla0001.corp.chartercom.com] [security-auditlog-2024.04.24] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[2024-04-24T07:30:00,403][INFO ][o.o.p.PluginsService     ] [vm0unelkdla0001.corp.chartercom.com] PluginService:onIndexModule index:[security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg]
[2024-04-24T07:30:00,412][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [vm0unelkdla0001.corp.chartercom.com] Detected cluster change event for destination migration
[2024-04-24T07:30:00,453][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [vm0unelkdla0001.corp.chartercom.com] Detected cluster change event for destination migration
[2024-04-24T07:30:00,473][INFO ][o.o.p.PluginsService     ] [vm0unelkdla0001.corp.chartercom.com] PluginService:onIndexModule index:[security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg]
[2024-04-24T07:30:00,479][INFO ][o.o.c.m.MetadataMappingService] [vm0unelkdla0001.corp.chartercom.com] [security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg] create_mapping
[2024-04-24T07:30:00,496][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [vm0unelkdla0001.corp.chartercom.com] Detected cluster change event for destination migration
[2024-04-24T07:30:06,992][INFO ][o.o.m.c.MLSyncUpCron     ] [vm0unelkdla0001.corp.chartercom.com] ML configuration already initialized, no action needed
[2024-04-24T07:30:46,977][INFO ][o.o.i.i.ManagedIndexCoordinator] [vm0unelkdla0001.corp.chartercom.com] Performing move cluster state metadata.
[2024-04-24T07:30:46,978][INFO ][o.o.i.i.MetadataService  ] [vm0unelkdla0001.corp.chartercom.com] ISM config index not exist, so we cancel the metadata migration job.
[2024-04-24T07:31:05,513][WARN ][o.o.s.a.BackendRegistry  ] [vm0unelkdla0001.corp.chartercom.com] Authentication finally failed for p3087732adm from 22.242.73.84:38996
[2024-04-24T07:31:05,523][INFO ][o.o.p.PluginsService     ] [vm0unelkdla0001.corp.chartercom.com] PluginService:onIndexModule index:[security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg]
[2024-04-24T07:31:05,530][INFO ][o.o.c.m.MetadataMappingService] [vm0unelkdla0001.corp.chartercom.com] [security-auditlog-2024.04.24/IU5MOry5SuK43n1XRivTBg] update_mapping [_doc]
[2024-04-24T07:31:05,557][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [vm0unelkdla0001.corp.chartercom.com] Detected cluster change event for destination migration
[2024-04-24T07:31:06,923][WARN ][o.o.s.a.BackendRegistry  ] [vm0unelkdla0001.corp.chartercom.com] Authentication finally failed for p3087732adm from 22.242.73.84:38996
[2024-04-24T07:31:46,978][INFO ][o.o.i.i.ManagedIndexCoordinator] [vm0unelkdla0001.corp.chartercom.com] Cancel background move metadata process.
[2024-04-24T07:31:46,979][INFO ][o.o.i.i.ManagedIndexCoordinator] [vm0unelkdla0001.corp.chartercom.com] Performing move cluster state metadata.
[2024-04-24T07:31:46,979][INFO ][o.o.i.i.MetadataService  ] [vm0unelkdla0001.corp.chartercom.com] Move metadata has finished.
[2024-04-24T07:34:46,773][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:34:46,982][INFO ][o.o.i.i.PluginVersionSweepCoordinator] [vm0unelkdla0001.corp.chartercom.com] Canceling sweep ism plugin version job
[2024-04-24T07:39:46,774][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:44:46,775][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:44:46,988][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Starting housekeeping task for auto refresh streaming jobs.
[2024-04-24T07:44:46,992][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Finished housekeeping task for auto refresh streaming jobs.
[2024-04-24T07:49:46,776][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:54:46,777][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:59:46,777][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T07:59:46,992][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Starting housekeeping task for auto refresh streaming jobs.
[2024-04-24T07:59:46,994][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Finished housekeeping task for auto refresh streaming jobs.
[2024-04-24T08:04:46,778][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T08:09:46,779][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T08:14:46,780][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T08:14:46,994][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Starting housekeeping task for auto refresh streaming jobs.
[2024-04-24T08:14:46,995][INFO ][o.o.s.s.c.FlintStreamingJobHouseKeeperTask] [vm0unelkdla0001.corp.chartercom.com] Finished housekeeping task for auto refresh streaming jobs.
[2024-04-24T08:19:46,780][INFO ][o.o.j.s.JobSweeper       ] [vm0unelkdla0001.corp.chartercom.com] Running full sweep
[2024-04-24T08:20:31,331][WARN ][o.o.s.a.BackendRegistry  ] [vm0unelkdla0001.corp.chartercom.com] Authentication finally failed for p3087732adm from 22.242.73.84:45882
[2024-04-24T08:20:35,820][WARN ][o.o.s.a.BackendRegistry  ] [vm0unelkdla0001.corp.chartercom.com] Authentication finally failed for p3087732 from 22.242.73.84:45882
[2024-04-24T08:20:47,503][WARN ][o.o.s.a.BackendRegistry  ] [vm0unelkdla0001.corp.chartercom.com] Authentication finally failed for P3087732ADM from 22.242.73.84:45882

I was trying to add DN name in

_meta:
  type: "nodesdn"
  config_version: 2

# Define nodesdn mapping name and corresponding values
 cluster1:
   nodes_dn:
       - CN=UX-RG-******,OU=Role,OU=Unix,OU=Groups,OU=SPECTRUM,DC=CORP,DC=CHARTERCOM,DC=com

But this also not working.

2

Hi @AbhiAbhishek,

You have enable_ssl: false(disabled), but you are training to use hosts: ldaps.

You will need to enable SSL or use ldap instead of ldaps.

best,
mj

3

Moreover, please check here how to configure the hostname and port of your Active Directory servers: Active Directory and LDAP - OpenSearch Documentation

Best,
mj

4

Hi @Mantas,

Thank you so much for the reply, I enabled ssl true & also tried ldaps to ldap, still its not working, getting the same error & warning.

With Regards,
Abhishek M

5

Hi @AbhiAbhishek,

I’ve just noticed that your config file is full of duplicate values (i.e: userbase, usersearch, etc… ), you will have to clean it up, please refer to:

Active Directory and LDAP - OpenSearch Documentation

best,
mj

6

Hi @Mantas
My config file resembles almost similar to documentation
which is

for aunthentication

ldap:
  http_enabled: true
  transport_enabled: true
  order: 1
  http_authenticator:
    type: basic
    challenge: true
  authentication_backend:
    type: ldap
    config:
      enable_ssl: true
      enable_start_tls: false
      enable_ssl_client_auth: false
      verify_hostnames: true
      hosts:
        - ldap.example.com:636
      bind_dn: cn=admin,dc=example,dc=com
      password: password
      userbase: 'ou=people,dc=example,dc=com'
      usersearch: '(sAMAccountName={0})'
      username_attribute: uid

for authorization:

authz:
  ldap:
    http_enabled: true
    transport_enabled: true
    authorization_backend:
      type: ldap
      config:
        enable_ssl: true
        enable_start_tls: false
        enable_ssl_client_auth: false
        verify_hostnames: true
        hosts:
          - ldap.example.com:636
        bind_dn: cn=admin,dc=example,dc=com
        password: password
        userbase: 'ou=people,dc=example,dc=com'
        usersearch: '(uid={0})'
        username_attribute: uid
        rolebase: 'ou=groups,dc=example,dc=com'
        rolesearch: '(member={0})'
        userroleattribute: null
        userrolename: none
        rolename: cn
        resolve_nested_roles: true
        skip_users:
          - kibanaserver
          - 'cn=Jane Doe,ou*people,o=TEST'
          - '/\S*/'

Which is also using multiple userbases.

So are you telling to use them only once?

With Regards,
Abhishek M

7

Hi @Mantas

Actually the opensearch service is not reading the config file under opensearch security I guess, making http challenge: true is not relecting in UI & also i tried applying only ldap configuration with order 0 its not picking up the latest one its still using default one. So what s the reason for this issue it tried reinstalling the opensearch also

With Regards,
Abhishek

8

Hi @AbhiAbhishek,

How are you applying these changes? Are you using securityadmin.sh?
Please see more here: Applying changes to configuration files - OpenSearch Documentation

Would you mind running the below and sharing the output (please blank if any sensitive information):

curl --insecure -u <admin_username>:<admin_password> -XGET https://<OS_node>:9200/_plugins/_security/api/securityconfig?pretty

Best,
mj

1 Like
9

@Mantas

Thank you for the reply, I am able to apply changes now with securityadmin.sh she’ll script & now I am able to login to the opensearch dashboard.

With Regards
Abhishek

1 Like
10

@Mantas

But not able to login to the opensearch url with ldap user it shows the user is not authorized, may I please know how to apply admin roles to the ldap group in role mapping. In ElastiSearch role mapping was easier just adding DN group name & respective roles in kibana & same with proper json format to run as Api.

But I am not getting to map the ldap like ElastiSearch in opensearch so please share me the role mapping for my ldap.
I don’t know how, where & need to add DN fully or not.

With Regards,
Abhishek

11

Hi @AbhiAbhishek,

Can you run the below and share the output:

curl --insecure -u <ldap_user>:<ldap_password> -XGET https://<OS_node>:9200/_plugins/_security/authinfo?pretty

This should show us what back-end roles are assigned to your LDAP user.

Best,
mj

12

Hi @Mantas

Thank you so much for the response. This is the output I got after the above mention command

{
  "user" : "User [name=p3087732adm, backend_roles=[], requestedTenant=null]",
  "user_name" : "p3087732adm",
  "user_requested_tenant" : null,
  "remote_address" : "22.242.73.84:54282",
  "backend_roles" : [ ],
  "custom_attribute_names" : [
    "attr.ldap.unixHomeDirectory",
    "attr.ldap.cacheModifiersName",
    "attr.ldap.objectGUID",
    "attr.ldap.userAccountControl",
    "ldap.original.username",
    "attr.ldap.prismuser",
    "attr.ldap.cacheCreatorsName",
    "attr.ldap.employeeType",
    "attr.ldap.sAMAccountName",
    "attr.ldap.uid",
    "attr.ldap.givenName",
    "ldap.dn",
    "attr.ldap.cn",
    "attr.ldap.cacheModifyTimestamp",
    "attr.ldap.description",
    "attr.ldap.gidNumber",
    "attr.ldap.name",
    "attr.ldap.cacheCreateTimestamp",
    "attr.ldap.uidNumber",
    "attr.ldap.displayName",
    "attr.ldap.objectSid",
    "attr.ldap.sn",
    "attr.ldap.cnmemberof",
    "attr.ldap.objectclass",
    "attr.ldap.uuid",
    "attr.ldap.loginShell"
  ],
  "roles" : [
    "own_index"
  ],
  "tenants" : {
    "p3087732adm" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}

So how can I assign roles to my ldap group to have complete admin access of the cluster.

With Regards,
Abhishek M

13

Hi @AbhiAbhishek,

You do not have any backend roles assigned to your user ( "user_name" : "p3087732adm"), how are you mapping your LDAP user to your OpenSearch internal roles?

best,
mj

14

@Mantas
This is the same issue that you assisted to me in different topic

I haven’t assigned any roles by rolemapping & i don’t know how to do it its not straight forward like elasticsearch likw in role mapping add DN group name & assign roles, I only setup the ldap integration whats above & I am directly able to login to the opensearch dashboard, which is not the use case in elasticsearch only after assigning certain roles to the ldap group we are able to login.

With Regards,
Abhishek M

15

Hi @AbhiAbhishek,

It is getting tricky to follow on these two separate forums.
Would you mind summarizing all in the new “fresh” forum?

Please include the latest of:

  • short description of accepted behaviour and the issue

  • config.yml

  • curl --insecure -u <admin_username>:<admin_password> -XGET https://<OS_node>:9200/_plugins/_security/api/securityconfig?pretty

  • curl --insecure -u <ldap_user>:<ldap_password> -XGET https://<OS_node>:9200/_plugins/_security/authinfo?pretty

  • ldapsearch on the ldap_user (if possible)

note: blank all sensitive information.

best,
mj