Getting weird error when trying to load saved objects in OSD 2.7.0

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

2.7.0

Describe the issue:

I am using Opensearch 2.7.0 and Opensearch dashboards 2.7.0.

I ran fresh Opensearch dashboard without importing anything from previous versions. When I go to saved objects, I get error:

Unable find saved objects

Tracing route of error: this request:

<KIBANA_HOST>/api/opensearch-dashboards/management/saved_objects/_find?perPage=50&page=1&fields=id&type=config&type=url&type=index-pattern&type=query&type=dashboard&type=visualization&type=visualization-visbuilder&type=search&sortField=type

is giving back:

{“statusCode”:400,“error”:“Bad Request”,“message”:“all shards failed: search_phase_execution_exception: [illegal_argument_exception] Reason: Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [type] in order to load field data by uninverting the inverted index. Note that this can use significant memory.”}

If I remove the last parameter (sortField=type), then the query goes through correctly.

This is what my .kibana index look like:

"hits" : [
      {
        "_index" : ".kibana",
        "_id" : "config:2.7.0",
        "_score" : 1.0,
        "_source" : {
          "config" : {
            "buildNum" : 6100
          },
          "type" : "config",
          "references" : [ ],
          "migrationVersion" : {
            "config" : "7.9.0"
          },
          "updated_at" : "2023-05-25T21:04:19.814Z"
        }
      }
    ]

Mapping:

{
  ".kibana" : {
    "mappings" : {
      "properties" : {
        "config" : {
          "properties" : {
            "buildNum" : {
              "type" : "long"
            }
          }
        },
        "migrationVersion" : {
          "properties" : {
            "config" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        },
        "type" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "updated_at" : {
          "type" : "date"
        }
      }
    }
  }
}

Configuration:

server.host: 0.0.0.0
opensearch.hosts: https://localhost:9200
opensearch.ssl.verificationMode: none
opensearch.username: <>
opensearch.password: <>
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
opensearch.requestHeadersAllowlist: [ authorization,securitytenant ]
opensearch_security.multitenancy.enabled: false
opensearch_security.multitenancy.tenants.enable_global: true
opensearch_security.multitenancy.tenants.enable_private: false
opensearch_security.multitenancy.tenants.preferred: ["Global"]
opensearch_security.multitenancy.enable_filter: true


#can't find replacement from es
#map.includeOpenSearchService: false
logging.dest: logs/kibana_server.log
logging.verbose: false

server.ssl.supportedProtocols: [TLSv1.2]
server.ssl.cipherSuites: [ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384]

opensearch_security.cookie.ttl: 86400000
opensearch_security.session.ttl: 86400000
opensearch_security.session.keepalive: true
opensearch_security.jwt.login_endpoint: /
opensearch_security.auth.type: "jwt"
opensearch_security.jwt.url_param: jwtToken

server.cors: true
server.customResponseHeaders:
  Access-Control-Allow-Credentials: "true"

Relevant Logs or Screenshots:

2

@richard24 Could you share your config.yml?
How did you deploy the cluster?

3

Thank you for response, the issue went away after we restarted a few times

1 Like
4

I am currently facing this same issue. This is my opensearch.yml.

I am deploying opensearch using helm to a cluster across 3 nodes all at once. 3 master nodes and 9 data nodes in total. I have yet to resolve this issue.

cluster.name: "my-opensearch"

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

node.attr.zone: "myzone3"

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# Implicitly done if ".singleNode" is set to "true".
# discovery.type: single-node

# Start OpenSearch Security Demo Configuration
# WARNING: revise all the lines below before you go into production
plugins:
  security:
    ssl:
      transport:
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
        enforce_hostname_verification: false
      http:
        enabled: true
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
    allow_unsafe_democertificates: true
    allow_default_init_securityindex: true
    authcz:
      admin_dn:
        - CN=kirk,OU=client,O=client,L=test,C=de
    audit.type: internal_opensearch
    enable_snapshot_restore_privilege: true
    check_snapshot_restore_write_privileges: true
    restapi:
      roles_enabled: ["all_access", "security_rest_api_access"]
    system_indices:
      enabled: true
      indices:
        [
          ".opendistro-alerting-config",
          ".opendistro-alerting-alert*",
          ".opendistro-anomaly-results*",
          ".opendistro-anomaly-detector*",
          ".opendistro-anomaly-checkpoints",
          ".opendistro-anomaly-detection-state",
          ".opendistro-reports-*",
          ".opendistro-notifications-*",
          ".opendistro-notebooks",
          ".opendistro-asynchronous-search-response*",
        ]
######## End OpenSearch Security Demo Configuration ########
5

The following suggestion fixed it for me: Kibana 7+ can't search saved_objects, FieldData error thrown on .kibana index · Issue #40133 · elastic/kibana · GitHub

I currently do not have the time to debug this behaviour. Maybe I’ll get back to it some day