Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Opensearch 2.4.0
OpenSearch Dashboards 2.4.0
OpenSearch Helm Chart 2.8.0/2.4.0
OpenSearch Dashboards Helm Chart 2.6.0/2.4.0
Describe the issue:
I’ve installed Opensearch via the official helm charts. Running 1.3.6 results in everything working just fine, meaning, dashboards can communicate with the master cluster
If I do an upgrade to 2.x.x everything also works fine! Dashboard gets updated and manages to communicate with the cluster. However, if I delete my PVs (meaning my storage) and thus do a fresh install, this does not seem to work and it gives the following very descriptive error message:
{"type":"log","@timestamp":"2022-11-28T15:53:41Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]:
Response Error"}
…this gets repeated forever.
These are the logs right before it fails:
{"type":"log","@timestamp":"2022-11-28T15:28:15Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"dataSourceManagement\" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]"}
{"type":"log","@timestamp":"2022-11-28T15:28:16Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"dataSource\" is disabled."}
{"type":"log","@timestamp":"2022-11-28T15:28:16Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2022-11-28T15:28:16Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"cpu.cgroup.path.override\" is deprecated and has been replaced by \"ops.cGroupOverrides.cpuPath\""}
{"type":"log","@timestamp":"2022-11-28T15:28:16Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"cpuacct.cgroup.path.override\" is deprecated and has been replaced by \"ops.cGroupOverrides.cpuAcctPath\""}
{"type":"log","@timestamp":"2022-11-28T15:28:17Z","tags":["warning","config","deprecation"],"pid":1,"message":"\"opensearch.requestHeadersWhitelist\" is deprecated and has been replaced by \"opensearch.requestHeadersAllowlist\""}
{"type":"log","@timestamp":"2022-11-28T15:28:19Z","tags":["info","plugins-system"],"pid":1,"message":"Setting up [49] plugins: [securityAnalyticsDashboards,alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,reportsDashboards,indexManagementDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,visBuilder,tileMap,regionMap,customImportMapDashboards,inputControlVis,ganttChartDashboards,visualize,searchRelevanceDashboards,queryWorkbenchDashboards,notificationsDashboards,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,observabilityDashboards,discover,savedObjectsManagement,securityDashboards,bfetch]"}
{"type":"log","@timestamp":"2022-11-28T15:28:28Z","tags":["info","savedobjects-service"],"pid":1,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations..."}
{"type":"log","@timestamp":"2022-11-28T15:28:28Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-11-28T15:28:29Z","tags":["error","savedobjects-service"],"pid":1,"message":"Unable to retrieve version information from OpenSearch nodes."}
On the master/data nodes, no error message gets send, so I’m guessing it doesn’t even try to connect. Is there a way to log which service it tries to connect to or at least get something more…verbose?
Configuration:
opensearch_dashboards.yml: |
server:
name: opensearch-dashboards
host: "0"
opensearch:
ssl:
verificationMode: certificate
certificate: /usr/share/opensearch/config/pki/dashboard.pem
key: /usr/share/opensearch/config/pki/dashboard.key
certificateAuthorities: [/usr/share/opensearch/config/pki/ca.pem]
requestHeadersWhitelist:
- "securitytenant"
- "Authorization"
- "x-forwarded-for"
- "x-proxy-user"
- "x-proxy-roles"
opensearch_security:
multitenancy:
enabled: true
tenants:
preferred:
- "Private"
- "Global"
readonly_mode:
roles: ["kibana_read_only"]
cookie:
secure: false
auth:
type: "proxy"
proxycache:
user_header: "x-proxy-user"
roles_header: "x-proxy-roles"
