Use dev tools get "internal server error" 500 error

OpenSearch Dashboards
1

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
opensearch 2.11.1
opensearch dashboard 2.11.0

Describe the issue:
when we use dev tools to exec some api and get “internal server error” with error code 500, but when we try to use some admin user to curl opensearch api in terminal, it make sense.

it looks like that internal user and role of dashboard dev tools have changed, but can I get some help to dive into?

This is a recent issue. Previously, the dev tools were functioning normally
Configuration:

Relevant Logs or Screenshots:
use api

image
image3224×348 151 KB

dev tool

image
image4168×408 61.3 KB

2

I noticed that your OpenSearch and OSD are of different versions (OS 2.11 vs. OSD 2.13 repsectively). Does the error resolve if you run the same version for each?

3

Sry, I checked our config again, and version is
OSD 2.11.0
OS 2.11.1
So maybe it will not be the problem?

4

I found some related issue and it looks like it is the frontend problem, However, since this is a production environment, I cannot modify the frontend code.

Is there any other way to fix this issue?

5

@pkingqaq Did you resolve your issue?

6

Actually not, now we can use some api tools like Postman to workaround
It seems usable, but it’s still not as convenient as “dev tools”.

7

@pkingqaq Could you share your opensearch_dashboards.yml and config.yml files?
Does it happen for admin user too?

Do you see any errors in OpenSearch Dashboards logs?

Did you test different browsers and private mode? Do you use any mod header in your browsers?

Does it fail for any API?

You shouldn’t assign admin permission to kibanaserver user. This is only a service account that should have only kibana_server role assigned.

root@docker4:/home/pablo# curl --insecure -u kibanaserver:kibanaserver https://localhost:9200/_plugins/_security/authinfo?pretty
{
  "user" : "User [name=kibanaserver, backend_roles=[], requestedTenant=null]",
  "user_name" : "kibanaserver",
  "user_requested_tenant" : null,
  "remote_address" : "172.18.0.1:49174",
  "backend_roles" : [ ],
  "custom_attribute_names" : [ ],
  "roles" : [
    "own_index",
    "kibana_server"
  ],
  "tenants" : {
    "kibanaserver" : true
  },
  "principal" : null,
  "peer_certificates" : "0",
  "sso_logout_url" : null
}
8

Each time I call the API via Dev Tools, I always receive error messages such as the ones shown below.

{
    "type": "error",
    "@timestamp": "2025-01-06T06:18:06Z",
    "tags":
    [],
    "pid": 1,
    "level": "error",
    "error":
    {
        "message": "Internal Server Error",
        "name": "Error",
        "stack": "Error: Internal Server Error\n    at HapiResponseAdapter.toInternalError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:80:19)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:177:34)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"
    },
    "url": "http://opensearch-dashboard.kup-ohio.metabit-trading.com/api/console/proxy?path=_mapping&method=GET",
    "message": "Internal Server Error"
}

{
    "type": "log",
    "@timestamp": "2025-01-06T06:18:06Z",
    "tags":
    [
        "error",
        "http"
    ],
    "pid": 1,
    "message": "Error: expected 'location' header to be set\n    at HapiResponseAdapter.toRedirect (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:123:13)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:105:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:92:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:164:34)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"
}

Did you test different browsers and private mode? Do you use any mod header in your browsers?

yes, I tried but not fine

Does it fail for any API?

yes, I try serveral apis but get the same error

9

@pkingqaq It looks like you’re connecting through reverse proxy to OpenSearch Dashboards.
How is OpenSearch Dashboards connects to OpenSearch nodes? Is it also through reverse proxy or directly.
Your curls are executed against direct OpenSearch endpoint https://localhost:9200.

10

@pkingqaq Also check for any additional errors in OpenSearch logs. HTTP 500 is usually preceded by other errors.

11

How to check the connection mode between dashboard and cluster node?
Here is my opensearch_dashboard.yml


logging.verbose: false
opensearch.hosts:
- "<<cluster.url>>"
opensearch.logQueries: false
opensearch.password: xxx
opensearch.requestHeadersWhitelist:
- authorization
- securitytenant
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch_security.auth.type: saml
opensearch_security.cookie.secure: true
opensearch_security.cookie.ttl: 216000000
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred:
- Private
- Global
opensearch_security.readonly_mode.roles:
- kibana_read_only
opensearch_security.session.keepalive: true
opensearch_security.session.ttl: 216000000
server.host: "0"
server.xsrf.whitelist:
- /_plugins/_security/saml/acs/idpinitiated
- /_plugins/_security/saml/acs
- /_plugins/_security/saml/logout
- /_opendistro/_security/saml/acs/idpinitiated
- /_opendistro/_security/saml/acs
- /_opendistro/_security/saml/logout