OpenSearch Dashboard not starting

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): opensearch-dashboards-2.14.0/Ubuntu 22.04.1

Describe the issue:

I have a fresh install of Opensearch 2.14.0, that I’m running directly from the upacked Tarball. It’s running nicely and I already have some indexes working.

However, Dashboards is not able to connect to Opensearch. I think the problem with the error “Unable to retrieve version information from OpenSearch nodes.”.

Configuration:

server.port: 5601
server.host: “0.0.0.0”
opensearch.hosts: [“https://localhost:9200”]
opensearch.ssl.verificationMode: none
opensearch.username: “admin”
opensearch.password: “my_passoword”
opensearch.requestHeadersWhitelist: [authorization, securitytenant]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]

## Use this setting if you are running opensearch-dashboards without https

opensearch_security.cookie.secure: false

Relevant Logs or Screenshots:

azureuser@demos-tax-customs:~/opensearch-dashboards-2.14.0$ ./bin/opensearch-dashboards
log [14:56:15.991] [info][plugins-service] Plugin “dataSourceManagement” has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
log [14:56:15.997] [info][plugins-service] Plugin “applicationConfig” is disabled.
log [14:56:15.997] [info][plugins-service] Plugin “cspHandler” is disabled.
log [14:56:15.998] [info][plugins-service] Plugin “dataSource” is disabled.
log [14:56:15.999] [info][plugins-service] Plugin “workspace” is disabled.
log [14:56:15.999] [info][plugins-service] Plugin “visTypeXy” is disabled.
log [14:56:16.044] [warning][config][deprecation] “opensearch.requestHeadersWhitelist” is deprecated and has been replaced by “opensearch.requestHeadersAllowlist”
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
log [14:56:16.186] [info][plugins-system] Setting up [52] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,securityAnalyticsDashboards,savedObjects,home,apmOss,reportsDashboards,searchRelevanceDashboards,dashboard,mlCommonsDashboards,visualizations,visTypeVega,visTypeTimeline,visTypeTable,visBuilder,visTypeMarkdown,visAugmenter,anomalyDetectionDashboards,alertingDashboards,tileMap,regionMap,customImportMapDashboards,inputControlVis,ganttChartDashboards,visualize,indexManagementDashboards,notificationsDashboards,management,indexPatternManagement,advancedSettings,console,dataExplorer,charts,visTypeTimeseries,visTypeVislib,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,securityDashboards,assistantDashboards,observabilityDashboards,queryWorkbenchDashboards,bfetch]
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
[agentkeepalive:deprecated] options.freeSocketKeepAliveTimeout is deprecated, please use options.freeSocketTimeout instead
log [14:56:16.768] [info][savedobjects-service] Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations…
log [14:56:17.121] [error][data][opensearch] [ResponseError]: Response Error
log [14:56:17.159] [error][savedobjects-service] Unable to retrieve version information from OpenSearch nodes.
log [14:56:19.612] [error][data][opensearch] [ResponseError]: Response Error
log [14:56:22.113] [error][data][opensearch] [ResponseError]: Response Error
log [14:56:24.615] [error][data][opensearch] [ResponseError]: Response Error
log [14:56:27.115] [error][data][opensearch] [ResponseError]: Response Error
log [14:56:29.616] [error][data][opensearch] [ResponseError]: Response Error

Hi @mwon

Could you please execute the following command?
curl --insecure -u "your_username":"your_password"-XGET https://localhost:9200

Also, please try changing requestHeadersWhitelist to the following:
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]

Sorry, forgot to inform that also tried a simple curl get and is working fine:

{
  "name" : "demos-tax-customs",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "qAmWq_FLTK-Q7CM-7nzwIg",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.14.0",
    "build_type" : "tar",
    "build_hash" : "aaa555453f4713d652b52436874e11ba258d8f03",
    "build_date" : "2024-05-09T18:51:00.973564994Z",
    "build_snapshot" : false,
    "lucene_version" : "9.10.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Tried with the new requestHeadersWhitelist, but got the same error.

Can you share the configuration file of OpenSearch node? It should be opensearch.yml

This my opensearch configuration:

*# Bind OpenSearch to the correct network interface. Use 0.0.0.0*

*# to include all available interfaces or specify an IP address*

*# assigned to a specific interface.*

network.host: 0.0.0.0

*# Unless you have already configured a cluster, you should set*

*# discovery.type to single-node, or the bootstrap checks will*

*# fail when you try to start the service.*

discovery.type: single-node

*# If you previously disabled the Security plugin in opensearch.yml,*

*# be sure to re-enable it. Otherwise you can skip this setting.*

plugins.security.disabled: false

plugins.security.ssl.transport.pemcert_filepath: /home/azureuser/opensearch-2.14.0/config/node1.pem

plugins.security.ssl.transport.pemkey_filepath: /home/azureuser/opensearch-2.14.0/config/node1-key.pem

plugins.security.ssl.transport.pemtrustedcas_filepath: /home/azureuser/opensearch-2.14.0/config/root-ca.pem

plugins.security.ssl.http.enabled: true

plugins.security.ssl.http.pemcert_filepath: /home/azureuser/opensearch-2.14.0/config/node1.pem

plugins.security.ssl.http.pemkey_filepath: /home/azureuser/opensearch-2.14.0/config/node1-key.pem

plugins.security.ssl.http.pemtrustedcas_filepath: /home/azureuser/opensearch-2.14.0/config/root-ca.pem

plugins.security.allow_default_init_securityindex: true

plugins.security.authcz.admin_dn:

**-** 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'

plugins.security.nodes_dn:

**-** 'CN=node1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'

plugins.security.audit.type: internal_opensearch

plugins.security.enable_snapshot_restore_privilege: true

plugins.security.check_snapshot_restore_write_privileges: true

plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]

Changed settings plugins.security.disabled: true
In opensearch.yml, try with these settings below:
http.port: 9200
transport.tcp.port: 9300
transport.publish_host: [“opensearch-node1:9300”]
discovery.seed_hosts: [“opensearch-node1:9300”]

In opensearch_dashboard.yml, try with these settings below:
opensearch.hosts: [“https://opensearch-node1:9200”]

Also, I suggest you should test without SSL config first, comment out below lines:

plugins.security.ssl.transport.pemcert_filepath: /home/azureuser/opensearch-2.14.0/config/node1.pem
plugins.security.ssl.transport.pemkey_filepath: /home/azureuser/opensearch-2.14.0/config/node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /home/azureuser/opensearch-2.14.0/config/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /home/azureuser/opensearch-2.14.0/config/node1.pem
plugins.security.ssl.http.pemkey_filepath: /home/azureuser/opensearch-2.14.0/config/node1-key.pem

plugins.security.ssl.http.pemtrustedcas_filepath: /home/azureuser/opensearch-2.14.0/config/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
- ‘CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA’

plugins.security.nodes_dn:

- ‘CN=node1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA’
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]

@mwon, could you please send the URLs of the tar files that you used to install OpenSearch Dashboards and OpenSearch?

@levi_1925 , now I have a new error, this time in opensearch execution:

BindTransportException[Failed to resolve publish address]; nested: UnknownHostException[opensearch-node1:9300: Name or service not known];

@Eugene7, these are the files I used:

https://artifacts.opensearch.org/releases/bundle/opensearch/2.14.0/opensearch-2.14.0-linux-x64.tar.gz
https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.14.0/opensearch-dashboards-2.14.0-linux-x64.tar.gz

And I have strictly followed the installation instructions (ignored step 3).