When i install S3 repository plugin in OpenSearch the OpenSearch not coming up

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.12.0
opensearch
-bash-4.2# curl https://172.31.4.236:9200 -u admin:admin -k
{
“name” : “ip-172-31-4-236.us-west-2.compute.internal”,
“cluster_name” : “opensearch”,
“cluster_uuid” : “S-7BjlTLS0Ot5VrY3ncgBQ”,
“version” : {
“distribution” : “opensearch”,
“number” : “2.12.0”,
“build_type” : “tar”,
“build_hash” : “2c355ce1a427e4a528778d4054436b5c4b756221”,
“build_date” : “2024-02-20T02:18:49.874618333Z”,
“build_snapshot” : false,
“lucene_version” : “9.9.2”,
“minimum_wire_compatibility_version” : “7.10.0”,
“minimum_index_compatibility_version” : “7.0.0”
},
“tagline” : “The OpenSearch Project: https://opensearch.org/
}

Describe the issue:
Every thing is fine in the CLI but dashboard not reached

Configuration:
opensearch dashboard output

-bash-4.2# cd opensearch-dashboards-2.12.0
-bash-4.2# curl -v 0.0.0.0:5601

  • Trying 0.0.0.0:5601…
  • Connected to 0.0.0.0 (127.0.0.1) port 5601

GET / HTTP/1.1
Host: 0.0.0.0:5601
User-Agent: curl/8.3.0
Accept: /

< HTTP/1.1 302 Found
< location: /app/login?
< osd-name: ip-172-31-4-236.us-west-2.compute.internal
< cache-control: private, no-cache, no-store, must-revalidate
< set-cookie: security_authentication=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:00:00 GMT; HttpOnly; Path=/
< content-length: 0
< Date: Mon, 18 Mar 2024 09:53:15 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<

  • Connection #0 to host 0.0.0.0 left intact

Relevant Logs or Screenshots:

@rathiga I will need a little more info here. How are you runing OpenSearch Dashboards? Also admin:admin is no longer a valid password. Its not strong enough and you will need to set a default password for the admin user that is strong

Hi @rathiga,

Could you please share your opensearch_dashboards.yml and opensearch.yml files?
How did you deploy your cluster?

Thanks,
mj

# opensearch.yml
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: localhost
network.host: 0.0.0.0
# Set a custom port for HTTP:
#network.host: 35.90.174.190
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
discovery.type: single-node
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the optimization of datetime formatters caching along with change in default datetime formatter
# Once there is no observed impact on performance, this feature flag can be removed.
#
#opensearch.experimental.optimization.datetime_formatter_caching.enabled: false
plugins.security.disabled: false
plugins.security.ssl.transport.pemcert_filepath: /opt/opensearch-2.12.0/config/node1.pem
plugins.security.ssl.transport.pemkey_filepath: /opt/opensearch-2.12.0/config/node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /opt/opensearch-2.12.0/config/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /opt/opensearch-2.12.0/config/node1.pem
plugins.security.ssl.http.pemkey_filepath: /opt/opensearch-2.12.0/config/node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /opt/opensearch-2.12.0/config/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.nodes_dn:
  - 'CN=node1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]

bash-4.2# cat opensearch_dashboards.yml

---
# Copyright OpenSearch Contributors
# SPDX-License-Identifier: Apache-2.0

# Description:
# Default configuration for OpenSearch Dashboards

# OpenSearch Dashboards is served by a back end server. This setting specifies t                                                                                                             he port to use.
server.port: 5601

# Specifies the address to which the OpenSearch Dashboards server will bind. IP                                                                                                              addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be ab                                                                                                             le to connect.
# To allow connections from remote users, set this parameter to a non-loopback a                                                                                                             ddress.
server.host: "0.0.0.0"

@rathiga, would you mind sharing your opensearch.yml as well?

Thanks,
mj

Hi @rathiga,

The opensearch_dashboards.yml seems to be missing some configuration, please see a sample below:


---

server.host: "0.0.0.0"
server.port: 5601

opensearch.hosts: [https://<OS_node>:9200]
opensearch.ssl.verificationMode: none
opensearch.username: <kibanaserver>
opensearch.password: <kibanaserver_password>
opensearch.requestHeadersWhitelist: [authorization]

opensearch_security.readonly_mode.roles: [kibana_read_only]
# Use this setting if you are running opensearch-dashboards without https
opensearch_security.cookie.secure: false

More details here: OpenSearch-Dashboards/config/opensearch_dashboards.yml at main · opensearch-project/OpenSearch-Dashboards · GitHub

Let me know if you have any further questions.

Best,
mj

opensearch.hosts: [https://localhost:9200]
opensearch.ssl.verificationMode: none
opensearch.username: admin
opensearch.password: admin
opensearch.requestHeadersWhitelist: [authorization, securitytenant]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [Private, Global]
opensearch_security.readonly_mode.roles: [kibana_read_only]

Use this setting if you are running opensearch-dashboards without https

opensearch_security.cookie.secure: false

Hi @rathiga,

Could you try running curl -XGET --insecure -u 'admin:<admin password>' 'https://<OS_node>:9200' and share the output?

Thanks,
mj

-bash-4.2# curl https://172.31.4.236:9200 -u admin:admin -k
{
“name” : “ip-172-31-4-236.us-west-2.compute.internal”,
“cluster_name” : “opensearch”,
“cluster_uuid” : “S-7BjlTLS0Ot5VrY3ncgBQ”,
“version” : {
“distribution” : “opensearch”,
“number” : “2.12.0”,
“build_type” : “tar”,
“build_hash” : “2c355ce1a427e4a528778d4054436b5c4b756221”,
“build_date” : “2024-02-20T02:18:49.874618333Z”,
“build_snapshot” : false,
“lucene_version” : “9.9.2”,
“minimum_wire_compatibility_version” : “7.10.0”,
“minimum_index_compatibility_version” : “7.0.0”
},
“tagline” : “The OpenSearch Project: https://opensearch.org/
}

Is the CLI and the browser (from the screenshot) on the same machine?

What protocol are you using in here HTTP or HTTPS?

How did you deploy your cluster?

Thanks,
mj

I try with both protocol HTTP & HTTPS showing same result

How did you deploy your cluster?

-bash-4.2# sudo systemctl status opensearch-2.12.0.service
● opensearch-2.12.0.service - OpenSearch 2.12.0
Loaded: loaded (/etc/systemd/system/opensearch-2.12.0.service; enabled; vendo r preset: disabled)
Active: active (running) since Mon 2024-04-01 16:27:25 UTC; 1 day 23h ago
Process: 30913 ExecStart=/opt/opensearch-2.12.0/bin/opensearch -d (code=exited , status=0/SUCCESS)
Main PID: 31126 (java)
CGroup: /system.slice/opensearch-2.12.0.service
└─31126 /opt/opensearch-2.12.0/jdk/bin/java -Xshare:auto -Dopensea…

Apr 01 16:27:23 ip-172-31-4-236.us-west-2.compute.internal systemd[1]: Starti…
Apr 01 16:27:25 ip-172-31-4-236.us-west-2.compute.internal systemd[1]: Starte…
Hint: Some lines were ellipsized, use -l to show in full.
-bash-4.2# curl https://172.31.4.236:9200 -u admin:admin -k
{
“name” : “ip-172-31-4-236.us-west-2.compute.internal”,
“cluster_name” : “opensearch”,
“cluster_uuid” : “S-7BjlTLS0Ot5VrY3ncgBQ”,
“version” : {
“distribution” : “opensearch”,
“number” : “2.12.0”,
“build_type” : “tar”,
“build_hash” : “2c355ce1a427e4a528778d4054436b5c4b756221”,
“build_date” : “2024-02-20T02:18:49.874618333Z”,
“build_snapshot” : false,
“lucene_version” : “9.9.2”,
“minimum_wire_compatibility_version” : “7.10.0”,
“minimum_index_compatibility_version” : “7.0.0”
},
“tagline” : “The OpenSearch Project: https://opensearch.org/
}

-bash-4.2# sudo systemctl status opensearch-dashboards-2.12.0.service
● opensearch-dashboards-2.12.0.service - OpenSearch-Dashboards-2.12.0
Loaded: loaded (/etc/systemd/system/opensearch-dashboards-2.12.0.service; dis abled; vendor preset: disabled)
Active: active (running) since Mon 2024-04-01 14:57:36 UTC; 2 days ago
Main PID: 32757 (node)
CGroup: /system.slice/opensearch-dashboards-2.12.0.service
└─32757 /opt/opensearch-dashboards-2.12.0/node/fallback/bin/node /…

Apr 01 16:27:25 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:27 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:30 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:32 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:35 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:37 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:40 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:27:43 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:28:34 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Apr 01 16:30:20 ip-172-31-4-236.us-west-2.compute.internal opensearch-dashboards [32757]: …
Hint: Some lines were ellipsized, use -l to show in full.
-bash-4.2# curl -v 0.0.0.0:5601

  • Trying 0.0.0.0:5601…
  • Connected to 0.0.0.0 (127.0.0.1) port 5601

GET / HTTP/1.1
Host: 0.0.0.0:5601
User-Agent: curl/8.3.0
Accept: /

< HTTP/1.1 302 Found
< location: /app/login?
< osd-name: ip-172-31-4-236.us-west-2.compute.internal
< cache-control: private, no-cache, no-store, must-revalidate
< set-cookie: security_authentication=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:0 0:00 GMT; HttpOnly; Path=/
< content-length: 0
< Date: Wed, 03 Apr 2024 16:00:55 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<

  • Connection #0 to host 0.0.0.0 left intact

log [15:56:13.862] [info][plugins-service] Plugin “dataSourceManagement” has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
log [15:56:13.869] [info][plugins-service] Plugin “dataSource” is disabled.
log [15:56:13.870] [info][plugins-service] Plugin “visTypeXy” is disabled.
log [15:56:13.927] [warning][config][deprecation] “opensearch.requestHeaders Whitelist” is deprecated and has been replaced by “opensearch.requestHeadersAllo wlist”
log [15:56:13.931] [fatal][root] InvalidConfigurationError: Unknown configur ation key(s): “opensearch_security.multitenancy.enabled”, “opensearch_security.m ultitenancy.tenants.preferred”, “opensearch_security.readonly_mode.roles”, “open search_security.cookie.secure”. Check for spelling errors and ensure that expect ed plugins are installed.
at ensureValidConfiguration (/opt/opensearch-dashboards-2.12.0/src/core/serv er/legacy/config/ensure_valid_configuration.js:50:11)
at Server.setup (/opt/opensearch-dashboards-2.12.0/src/core/server/server.js :156:5)
at Root.setup (/opt/opensearch-dashboards-2.12.0/src/core/server/root/index. js:60:14)
at bootstrap (/opt/opensearch-dashboards-2.12.0/src/core/server/bootstrap.js :113:5)
at Command. (/opt/opensearch-dashboards-2.12.0/src/cli/serve/serv e.js:192:5) {
code: ‘InvalidConfig’,
processExitCode: 64,
cause: undefined
}

FATAL Error: Unknown configuration key(s): “opensearch_security.multitenancy.e nabled”, “opensearch_security.multitenancy.tenants.preferred”, “opensearch_secur ity.readonly_mode.roles”, “opensearch_security.cookie.secure”. Check for spellin g errors and ensure that expected plugins are installed.

There is a blank (space) in “e nabled”, you need to fix it and try again.

best,
mj

Could you please share your, opensearch.yml and opensearch_dashboards.yml?

Thanks,
mj

-bash-4.2# curl -v 127.0.0.1:5601

  • Trying 127.0.0.1:5601…
  • Connected to 127.0.0.1 (127.0.0.1) port 5601

GET / HTTP/1.1
Host: 127.0.0.1:5601
User-Agent: curl/8.3.0
Accept: /

< HTTP/1.1 302 Found
< location: /app/login?
< osd-name: ip-172-31-4-236.us-west-2.compute.internal
< cache-control: private, no-cache, no-store, must-revalidate
< set-cookie: security_authentication=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:00:00 GMT; HttpOnly; Path=/
< content-length: 0
< Date: Thu, 25 Apr 2024 13:05:09 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<

  • Connection #0 to host 127.0.0.1 left intact