Tenant Indices Don't Work As Expected - Prevents Reporting

Security
,
1

Background:

I am running OpenSearch v 1.2.4 and OpenSearch Dashboards v 1.2.0 using both Docker Swarm and Docker Compose (the behavior described below is identical, regardless of whether I am using swarm or compose). I am using OpenSearch’s docker images (both versions are the “latest” at the time I am writing this post) and copying my configuration files and certificates into the images.

I have configured OpenSearch and OpenSearch Dashboards to use OpenID Connect (Keycloak) for authentication and I have multitenancy enabled. I have no problem logging in using OpenID Connect, and my user is granted “all_access” rights based on my “admin” backend role.

Problem

I am able to log in and create dashboards; however, I am seeing some odd behavior:

  1. When I attempt to generate reports in the Global Tenant, I see an error in my console that says “no such index ‘.kibana’”. If I create an index alias to point the alias “.kibana” to “.opensearch_dashboards*”, it resolves this issue
  2. Even after applying the alias above, I cannot generate reports from a Private tenant. In my console, I see an error that says “no such index ‘.kibana_129834-myusername’”

A couple of additional things that I tried (I completely rebuilt images after each chang before testing) that did not resolve the issue:

  • Changing the name of the configuration field in config.yml from “kibana” to either “opensearch-dashboards” or “opensearch_dashboards” results in an UnrecognizedPropertyException when I start up opensearch
  • Leaving the configuration field as “kibana” and changing the lower “index” field from “.kibana” to “.opensearch_dashboards” still results in errors in the console that say “no such index ‘.kibana’”, even when I don’t have “.kibana” anywhere in any of my configuration files. Furthermore, if I set “index” to “.opensearch_dashboards”, I receive the same errors in the console but cannot even apply the alias fix to point “.kibana” to “.opensearch_dashboards” as described above.

Below is my security config.yml

config:
  dynamic:
    # Set filtered_alias_mode to 'disallow' to forbid more than 2 filtered aliases per index
    # Set filtered_alias_mode to 'warn' to allow more than 2 filtered aliases per index but warns about it (default)
    kibana: # Changing this to "opensearch-dashboards" or "opensearch_dashboards" results in UnrecognizedPropertyException
    # Kibana multitenancy
      multitenancy_enabled: true
      server_username: kibanaserver
      index: '.kibana' # Changing this to ".opensearch_dashboards" does not resolve issue and prevents ".kibana" index alias workaround described above
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: true
        #internalProxies: '192\.168\.0\.10|192\.168\.0\.11' # regex pattern
        internalProxies: '.*' # trust all internal proxies, regex pattern
        remoteIpHeader:  'x-forwarded-for'
        ###### see https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html for regex help
        ###### more information about XFF https://en.wikipedia.org/wiki/X-Forwarded-For
        ###### and here https://tools.ietf.org/html/rfc7239
        ###### and https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Remote_IP_Valve
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: http://keycloak:8100/auth/realms/poc/.well-known/openid-configuration
            jwks_uri: http://keycloak:8100/auth/realms/poc/protocol/openid-connect/certs
            openid_connect_idp.enable_ssl: false
            openid_connect_idp.verify_hostnames: false
            skip_users:
              - kibanaro
              - kibanaserver
              - logstash
              - adminp
              - kibanauser
              - filebeat_internal
        authentication_backend:
          type: noop

Questions

  • Is there a configuration key other than “kibana” that I should be setting in my security config.yml?
  • Is there another value for “index” that I should be setting in my security config.yml?
  • Any ideas why Opensearch Dashboards is looking for “.kibana” indices even in the cases when the string “.kibana” doesn’t appear anywhere in any of my configuration files?
2

@ryan.stewart113 As per documentation the multitenancy is enabled by default.

There is no need to configure it in config.yml. Can you try to comment them out and restart the stack.
OpenSearch still creates .kibana index by default.

3

@pablo Thanks for the suggestion! It wasn’t the multitenancy configuration, but it got me wondering why the .kibana index was not being created automatically at initialization. I noticed that all of the created indices were of the format .opensearch_dashboards_#. Turns out that it was the following setting in my opensearch_dashboards.yml that was preventing the .kibana indices from being created and managed correctly for tenants:

#opensearch_dashboards.yml
opensearchDashboards.index: ".opensearch_dashboards"

Commenting out this line and restarting resulted in the .kibana indices being created and managed correctly.

Many thanks!

1 Like
4

This also fixed my issue, Thx