Upgrade opensearch + opensearch dashboards to 2.14.0

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
Upgrading from 2.4.0 to 2.14.0

Describe the issue:
Hi, I’m trying to install clean 2.14.0 with the similar settings i’ve been using in 2.4.0

The opensearch goes up successfully, but the opensearch-dashboards process keeps getting ‘terminated’ everytime without an error.

I can rule out machine issues, I’ve enough memory and CPU, and same issue reproduce on few machines.

so its configuration issue.

Configuration:

  • Be aware I’ve took the settings from 2.4.0 environment i’m using, and it works there.
    So something changed in the settings between 2.4.0 and 2.14.0 that i’m missing,
    I’ve tried to update the plugins to 2.14.0 but the rest I left the same.
    If you have any idea, please let me know.

opensearch.yml

######## Start OpenSearch Configuration ########
plugins.security.ssl.transport.pemcert_filepath: certificates/elk-transport-crt.pem
plugins.security.ssl.transport.pemkey_filepath: certificates/elk-transport-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/elastic_ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: certificates/elk-node-crt.pem
plugins.security.ssl.http.pemkey_filepath: certificates/elk-node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/elastic_ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:

  • ‘CN=CONTROL-M_EM_ES_admin,O=ChangeMe,L=ChangeMeL,ST=ChangeMeST,C=CM’

Plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
.plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
.plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
.plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
.opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
.opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-,
.opensearch-notifications-
, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
.opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
.geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
.plugins-flow-framework-state]
node.max_local_storage_nodes: 3
######## End OpenSearch Configuration ########path.logs: /home/dbauser/ctm_em/log/services/workflow_insights
cluster.name: workflow_insights_cluster
network.host: 0
node.name: dba-tlv-woenwc
node.roles: [“initial_master”,“data”,“ingest”,“master”,“remote_cluster_client”]
action.auto_create_index: false
bootstrap.memory_lock: true
http.port: 19200
transport.port: 19300
cluster.initial_master_nodes: [‘dba-tlv-woenwc’]
discovery.seed_hosts: [‘dba-tlv-woenwc’]
plugins.security.nodes_dn:

  • ‘CN=CONTROL-M_EM_ES_transport,O=ChangeMe,L=ChangeMeL,ST=ChangeMeST,C=CM’

opensearch-dashboards.yml


server.port: 15061
server.host: ‘0.0.0.0’
opensearch.hosts: [‘https://localhost:19200’]
opensearch.ssl.verificationMode: certificate
OpenSearch.requestHeadersWhitelist: [“securitytenant”,“Authorization”]

opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: [“Global”]
opensearch_security.multitenancy.tenants.enable_private: false
opensearch_security.readonly_mode.roles: [“kibana_read_only”,“kibana_read”]
opensearch_security.cookie.secure: false

Use this setting if you are running opensearch-dashboards without https

#server.rewriteBasePath: true
server.basePath: “/services-proxy/kibana”
csp.warnLegacyBrowsers: false
csp.rules:

  • “script-src ‘unsafe-eval’ ‘self’ ‘unsafe-inline’”
    Options for the logging rotate feature
    logging.rotate:
    enabled: true
    everyBytes: 10000000
    keepFiles: 5
    usePolling: true
    pollingInterval: 10000

#logging.quiet : true # Remove log
#logging.verbose: true # Debug mode log
opensearch.ssl.certificateAuthorities: [‘/home/dbauser/ctm_em/ini/ssl/elastic_ca.pem’]
logging.dest: /home/dbauser/ctm_em/log/services/workflow_insights/opensearch-dashboards.log
server.ssl.enabled: true
server.ssl.key: /home/dbauser/ctm_em/opensearch-dashboards/config/certificates/kibana-node-key.pem
server.ssl.certificate: /home/dbauser/ctm_em/opensearch-dashboards/config/certificates/kibana-node-crt.pem

Relevant Logs or Screenshots:

Hi @nap608,

Did I understand correctly, that the OpenSearch-dashboards never start successfully?
Do you see any error messages in the logs (have you tried enabling logging.verbose: true)?

How did you deploy your cluster?

Thanks,
mj

I found why it was terminating and it wasn’t related to opensearch,
but i’m still getting errors on opensearch-dashboards:
deploy I just extract the tar and run the bin.
and i did set verbose true.

Hi @nap608,

how are you connecting (authenticating) from your OpenSearch dashboards to your OpenSearch cluster?

i.e:

opensearch.username: kibanaserver
opensearch.password: kibanaserver

best,
mj

Issue resolved, nevermind, thank you