Plugin is not working after upgrade from 2.9.0 to 2.11.0

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

2.11.0

Describe the issue:
I have upgraded OSD from v2.9.0 to v2.11.0 and my plugin have stopped working which was compatible with node version 16. I was observing some security related issues in the log though I’d not deployed the opensearch with security enabled.

Relevant Logs or Screenshots:

Index Management - CommonService - apiCaller StatusCodeError: no handler found for uri [/_plugins/_security/api/account] and method [GET]
at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
at HttpConnector. (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
at IncomingMessage.emit (node:events:525:35)
at IncomingMessage.emit (node:domain:489:12)
at endReadableNT (node:internal/streams/readable:1359:12)
at processTicksAndRejections (node:internal/process/task_queues:82:21) {
status: 400,
displayName: ‘BadRequest’,
path: ‘/_plugins/_security/api/account’,
query: undefined,
body: {
error: ‘no handler found for uri [/_plugins/_security/api/account] and method [GET]’
},
statusCode: 400,
response: ‘{“error”:“no handler found for uri [/_plugins/_security/api/account] and method [GET]”}’,
toString: [Function (anonymous)],
toJSON: [Function (anonymous)]

@shubtiwa This API is still valid in 2.11.0

Did you disable the security plugin?

Could you share your opensearch.yml file?

Yes I’d deployed without security.

opensearch.yml

======================== OpenSearch Configuration =========================

NOTE: OpenSearch comes with reasonable defaults for most settings.

Before you set out to tweak and tune the configuration, make sure you

understand what are you trying to accomplish and the consequences.

The primary way of configuring a node is via this file. This template lists

the most important settings you may want to configure for a production cluster.

Please consult the documentation for further information on configuration options:

https://www.opensearch.org

---------------------------------- Cluster -----------------------------------

Use a descriptive name for your cluster:

#cluster.name: my-application

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

#node.name: node-1

Add custom attributes to the node:

#node.attr.rack: r1

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/lib/opensearch

Path to log files:

path.logs: /var/log/opensearch

----------------------------------- Memory -----------------------------------

Lock the memory on startup:

#bootstrap.memory_lock: true

Make sure that the heap size is set to about half the memory available

on the system and that the owner of the process is allowed to use this

limit.

OpenSearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

Set the bind address to a specific IP (IPv4 or IPv6):

#network.host: 192.168.0.1

Set a custom port for HTTP:

#http.port: 9200

For more information, consult the network module documentation.

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when this node is started:

The default list of hosts is [“127.0.0.1”, “[::1]”]

#discovery.seed_hosts: [“host1”, “host2”]

Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:

#cluster.initial_cluster_manager_nodes: [“node-1”, “node-2”]

For more information, consult the discovery and cluster formation module documentation.

---------------------------------- Gateway -----------------------------------

Block initial recovery after a full cluster restart until N nodes are started:

#gateway.recover_after_nodes: 3

For more information, consult the gateway module documentation.

---------------------------------- Various -----------------------------------

Require explicit names when deleting indices:

#action.destructive_requires_name: true

---------------------------------- Remote Store -----------------------------------

Controls whether cluster imposes index creation only with remote store enabled

cluster.remote_store.enabled: true

Repository to use for segment upload while enforcing remote store for an index

node.attr.remote_store.segment.repository: my-repo-1

Repository to use for translog upload while enforcing remote store for an index

node.attr.remote_store.translog.repository: my-repo-1

---------------------------------- Experimental Features -----------------------------------

Gates the visibility of the experimental segment replication features until they are production ready.

OpenSearch.experimental.feature.segment_replication_experimental.enabled: false

Gates the functionality of a new parameter to the snapshot restore API

that allows for creation of a new index type that searches a snapshot

directly in a remote repository without restoring all index data to disk

ahead of time.

OpenSearch.experimental.feature.searchable_snapshot.enabled: false

Gates the functionality of enabling extensions to work with OpenSearch.

This feature enables applications to extend features of OpenSearch outside of

the core.

OpenSearch.experimental.feature.extensions.enabled: false

Gates the concurrent segment search feature. This feature enables concurrent segment search in a separate

index searcher threadpool.

OpenSearch.experimental.feature.concurrent_segment_search.enabled: false

######## Start OpenSearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:

  • CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [“.plugins-ml-config”, “.plugins-ml-connector”, “.plugins-ml-model-group”, “.plugins-ml-model”, “.plugins-ml-task”, “.plugins-ml-conversation-meta”, “.plugins-ml-conversation-interactions”, “.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opensearch-notifications-”, “.opensearch-notebooks”, “.opensearch-observability”, “.ql-datasources”, “.opendistro-asynchronous-search-response*”, “.replication-metadata-store”, “.opensearch-knn-models”, “.geospatial-ip2geo-data*”]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

@shubtiwa Just to confirm, you’ve stated that you’ve deployed the cluster without security but your opensearch.yml says the opposite.

plugins.security.ssl.http.enabled: true

Which one is correct then?

Could you share the output of the below command?

/usr/share/opensearch/bin/opensearch-plugin list

bash-4.4$ /usr/share/opensearch/bin/opensearch-plugin list
ingest-attachment
opensearch-cross-cluster-replication
opensearch-custom-codecs
opensearch-index-management
opensearch-job-scheduler
opensearch-reports-scheduler
opensearch-security
prometheus-exporter

@shubtiwa Could you share the opensearch-dashboards.yml file and the output of the below command?

curl --insecure -u admin:admin -XGET https://localhost:9200/_plugins/_security/api/securityconfig?pretty

@pablo really sorry ! , Please find the correct configuration files.

opensearch.yml

cluster:
name: ${CLUSTER_NAME}
initial_cluster_manager_nodes: ${CLUSTER_INITIAL_CLUSTER_MANAGER_NODES}

node:
name: ${NODE_NAME}
roles: ${NODE_ROLES}

network.host: ${NETWORK_HOST}

path:
data: /data/data
logs: /data/log
repo: ${PATH_REPO}

#bootstrap.memory_lock: true
http:
compression: true
port: ${HTTP_PORT}
cors:
enabled: ${HTTP_CORS_ENABLE}
allow-origin: ${HTTP_CORS_ALLOW_ORIGIN}

discovery:
seed_hosts: ${DISCOVERY_SERVICE}

transport:
port: ${TRANSPORT_PORT}

plugins.security.disabled: true
compatibility.override_main_response_version: true

opensearch-dashboards.yml

Donot change sever name and host. This is default configuration.

server.name: dashboards
server.customResponseHeaders: { “X-Frame-Options”: “DENY” }
#Set it to ‘true’ to help prevent the browser from allowing unsafe scripting. If true, it will block access to dashboards for any browser that does not enforce even a rudimentary set of Content Security Policy protections.
csp.strict: true
#Enable server.ssl.supportedProtocols when Security is enabled.
#server.ssl.supportedProtocols: [“TLSv1.2”]
OpenSearch security cookie can be secured by setting the below parameter to true. Uncomment it when Security is enabled.
#opensearch_security.cookie.secure: true
#To enable multitenancy in dashboards uncomment and set “opensearch_security.multitenancy.enabled” to true and uncomment opensearch.requestHeadersAllowlist parameter.
#opensearch_security.multitenancy.enabled: false

Allowlist basic headers and multi tenancy header

OpenSearch.requestHeadersAllowlist: [ “Authorization”, “securitytenant”, “x-forwarded-for”, “x-proxy-user”, “x-proxy-roles” ]
#opensearch_security.auth.unauthenticated_routes:

To use client-cert authentication for dashboards backend server with opensearch, enable clientcert_auth_domain in indexsearch and uncomment the two parameters “opensearch.ssl.certificate” and “opensearch.ssl.key”. Their values will be set internally. Add the required certificates in pre-created secret and configure them under sslsecretvolume section.

OpenSearch.ssl.certificate: “”
OpenSearch.ssl.key: “”

uncomment below section for keycloak authentication and provide required correct parameters

When sensitiveInfoInSecret is true then uncomment below section for keycloak authentication and set opensearch_security.openid.client_id: , opensearch_security.openid.client_secret: , opensearch_security.openid.root_ca: to empty so that they are internally populated from the secrets.

#opensearch_security.auth.type: “openid”
#opensearch_security.openid.connect_url: “https://:/auth/realms//.well-known/openid-configuration”
#opensearch_security.openid.client_id: “”
#opensearch_security.openid.client_secret: “”
#opensearch_security.openid.header: “Authorization”

for dashboards service on ingress port is not required

#opensearch_security.openid.base_redirect_url: “https://:/”

Do not change root_ca file path as this is the default mount path.

#opensearch_security.openid.root_ca: “/etc/opensearch-dashboards/certs/keycloakRootCaPem”

Uncomment below section for sane sso and provide required correct parameters

#csan.enabled: “true”
#csan.ssoproxy.url: https://:
#opendistro_security.auth.unauthenticated_routes: [“/api/status”, “/csan/v1/sso”]
#csan.sco.url: http://system-credential-orchestrator-svc.credential.svc.cluster.local
#csan.sco.keycloak_entity: “ckey”
#csan.sco.keycloak_classifier: “realm-admin”
#csan.sco.sane_entity: “sane”
#csan.sco.sane_plugin_name: “sane-credential-plugin”

@shubtiwa The shared error is expected.

/_plugins/_security/api/account

This API belongs to the security plugin. Since it is disabled in the opensearch.yml the OpenSearch doesn’t understand the API call.