Internal users unable to access dashboards

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch: 2.8.0
Dashboards: 2.8.0

Describe the issue:
After adding users to the internal_users.yml file they are able to access OpenSearch via the API, but when attempting to login via OppenSearch Dashboards an invalid username/password response is given.

If I login to the dashboard via an LDAP authenticated user, I can see that the internal users are present in the dashboard.

The kibanaserver user is able to login, but this is the default user

Configuration:
Opensearch.yml:


# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: {{ ansible_hostname }}{{ resolve_host }}

plugins.security.ssl.transport.pemcert_filepath: {{ public_crt_filename }}
plugins.security.ssl.transport.pemkey_filepath: {{ private_key_filename }}
plugins.security.ssl.transport.pemtrustedcas_filepath: {{ root_ca_filename }}
plugins.security.ssl.transport.enforce_hostname_verification: true
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: {{ public_crt_filename }}
plugins.security.ssl.http.pemkey_filepath: {{ private_key_filename }}
plugins.security.ssl.http.pemtrustedcas_filepath: {{ root_ca_filename }}
plugins.security.allow_unsafe_democertificates: false
plugins.security.allow_default_init_securityindex: true
plugins.security.nodes_dn:
  - '{{ cert_dn }}'
plugins.security.authcz.admin_dn:
  - '{{ cert_dn }}'

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models"]
node.max_local_storage_nodes: 3

path.repo: ["/mnt"]

internal_users.yml:

_meta:
  type: "internalusers"
  config_version: 2

dr_user:
  hash: "$2y$12$8RJNGU0K78e5BGt2pA2txuemnUDiGDai1CaZHR5LJdd7GWP2YEfoG"
  backend_roles:
    - "admin"
    - "kibanauser"
  description: "Disaster Recovery User"

verify_user:
  hash: "$2y$12$gGcT4KqwoIdyUarJWOG.G.xyuQwXh8WRp79rMzGo2BHU9Ilc.wdgW"
  backend_roles:
    - "readall"
  description: "User for verification/smoketests"

deploy_user:
  hash: "$2y$12$fCQ9S0v6HMVSFG6wzEy.iOd5fIKnAqeZGciiRTL72aoUmsZUKAC9G"
  backend_roles:
    - "admin"
  description: "Used to configured OpenSearch"

kibanaserver:
  hash: "$2a$12$4AcgAt3xwOWadA5s5blL6ev39OXDNhmOesEoo33eZtrq2N0YrU3H."
  reserved: true
  description: "Demo kibanaserver user"

opensearch_dashboards.yml:

opensearch.hosts: '[{{ hosts }}]'
opensearch.ssl.verificationMode: full
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: [authorization, securitytenant]
server.ssl.enabled: true
server.ssl.certificate: /usr/share/opensearch-dashboards/config/{{ public_crt_filename }}
server.ssl.key: /usr/share/opensearch-dashboards/config/{{ private_key_filename }}
opensearch.ssl.certificateAuthorities: ["/usr/share/opensearch-dashboards/config/{{ root_ca_filename }}"]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]
opensearch_security.cookie.secure: true
server.host: {{ groups['search_dashboard'][0] }}

role_mapping.yml:

_meta:
  type: "rolesmapping"
  config_version: 2

all_access:
  reserved: true
  backend_roles:
  - "admin"
  - "developers"
  description: "Maps admin to all_access"

own_index:
  reserved: false
  users:
  - "*"
  description: "Allow full access to an index named like the username"

readall:
  reserved: false
  backend_roles:
  - "readall"

manage_snapshots:
  reserved: false
  backend_roles:
  - "snapshotrestore"

kibana_user:
  reserved: false
  backend_roles:
  - "kibanauser"
  description: "Maps kibanauser to kibana_user"

kibana_server:
  reserved: true
  users:
  - "kibanaserver"

I am able to hit the API with my internal users:

curl --insecure -u "dr_user:pass" -XGET "https://$HOSTNAME:9200/_opendistro/_security/authinfo?pretty"
{
  "user" : "User [name=dr_user, backend_roles=[developer, all_access, kibanauser], requestedTenant=null]",
  "user_name" : "dr_user",
  "user_requested_tenant" : null,
  "remote_address" : "172.30.3.1:39998",
  "backend_roles" : [
    "developer",
    "all_access",
    "kibanauser"
  ],
  "custom_attribute_names" : [ ],
  "roles" : [
    "own_index",
    "kibana_user"
  ],
  "tenants" : {
    "global_tenant" : true,
    "dr_user" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}

The only part that stands out to me is that “all_access” is missing from roles, whereas on my LDAP user it shows:

  ],
  "roles" : [
    "own_index",
    "all_access"
  ],
  "tenants" : {
    "me@me.com" : true,
    "global_tenant" : true,
    "admin_tenant" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}

Hi @baazzaar,

The all_access should be assigned to your dr_user user because of the backend_roles: "admin", the internal user setup looks correct. Could you please share your config.yml file as well?

Thanks,
Mantas

Hi @Mantas,

Thanks for the reply.

Here is my config.yml:

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      internal_auth:
        order: 0
        description: "HTTP basic authentication using the internal user database"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      ldap_auth:
        order: 1
        description: "Authenticate using LDAP"
        http_enabled: true
        transport_enabled: true
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: ldap
          config:
            enable_ssl: true
            # Start TLS cannot be used in conjunction with LDAPS
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            pemtrustedcas_filepath: /usr/share/opensearch/config/trusted_cas.pem
            hosts:
              - ad.net:636
            bind_dn: ou=not staff,dc=office,dc=business,dc=net
            password: {{ lookup('secretserver_rest', 452, field='Password') }}
            userbase: ou=staff,dc=office,dc=business,dc=net
            usersearch: (sAMAccountName={0})
            username_attribute: userPrincipalName

    authz:
      ldap_roles:
        description: "Authorize using LDAP"
        http_enabled: true
        transport_enabled: true
        authorization_backend:
          type: ldap
          config:
            enable_ssl: true
            # Start TLS cannot be used in conjunction with LDAPS
            enable_start_tls: false
            enable_ssl_client_auth: false
            verify_hostnames: true
            pemtrustedcas_filepath: /usr/share/opensearch/config/trusted_cas.pem
            hosts:
              - ad.net:636
            bind_dn: cn=ou=not staff,dc=office,dc=business,dc=net
            password: password
            userbase: ou=staff,dc=office,dc=business,dc=net
            usersearch: '(sAMAccountName={0})'
            skip_users:
              - admin
              - kibanaserver
            rolebase: ou=staff,dc=office,dc=business,dc=net
            rolesearch: (member={0})
            userroleattribute: null
            userrolename: disabled
            rolename: cn
            resolve_nested_roles: false

Could you add dr_user to the skip_users list and give it a test run?

That seems to have done the trick! Thank you! :slight_smile:

One last thing, if you know the answer, does there always have to be a username and password specified in the opensearch_dashboards.yml file, or is there a way to obfuscate this?

You are welcome!

I did a bit of research on using certs instead of username and password but it does not look to be supported at the moment: [Feature Request] Authenticate Opensearch Dashboard users with PKI · Issue #1470 · opensearch-project/security · GitHub

You could remove the password after the OpenSearch Dashboards are booted up, however, it will have to be populated back in if any reboot.