Hi! I am new to opensearch!
Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):
- OpenSearch/ Detected OpenSearch Version: x-content-2.10.0 (from the logs with
docker logs opensearch-node1) - Dashboard/ wasn’t able to find it with
docker logs opensearch-dashboards - Server OS/ MacOS Big Sur
- Browser: NaN
Describe the issue:
I am trying to install OpenSearch from the official documentation but I have several problems and warnings. The errors are OpenSearch connection errors: OpenSearch Dashboards cannot connect to OpenSearch due to a connection error (ECONNREFUSED). This could be due to a problem in the configuration.
- I don’t think it is due to the fact that OpenSearch is not running because it must initialize after it is installed.
- I don’t think it is because of the RAM because it is indicated Memory: 4 GB
- I don’t think it is a problem in the configuration because the docker-compose.yaml comes from the official documentation.
Here are the warnings, if they may be the reason:
-
Obsolete tool warning: a warning is displayed informing that a tool will be discontinued in future versions of OpenSearch.A link is provided for more information. Warning about use on public systems: Warns that the security installer should not be used on public or production systems.
-
Security Warning: Warning about the use of an obsolete method in java.lang.System. It is suggested to report this issue to the OpenSearch maintainers, so, here I am!
-
Insecure directory permissions warnings: Warnings issued about insecure permissions on a directory in the OpenSearch configuration. This could pose a security risk?
Configuration:
The one provided here:
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
Relevant Logs or Screenshots:
Not sure what is a relevant log …
(base) remplacement@remplacements-MacBook-Pro Downloads % docker-compose up
[+] Running 3/0
✔ Container opensearch-dashboards C... 0.0s
✔ Container opensearch-node2 Create... 0.0s
✔ Container opensearch-node1 Create... 0.0s
Attaching to opensearch-dashboards, opensearch-node1, opensearch-node2
opensearch-node1 | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch-node2 | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch-node1 | **************************************************************************
opensearch-node1 | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch-node1 | ** https://github.com/opensearch-project/security/issues/1755 **
opensearch-node1 | **************************************************************************
opensearch-node2 | **************************************************************************
opensearch-node2 | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch-node2 | ** https://github.com/opensearch-project/security/issues/1755 **
opensearch-node2 | **************************************************************************
opensearch-node1 | OpenSearch Security Demo Installer
opensearch-node1 | ** Warning: Do not use on production or public reachable systems **
opensearch-node2 | OpenSearch Security Demo Installer
opensearch-node2 | ** Warning: Do not use on production or public reachable systems **
opensearch-node2 | Basedir: /usr/share/opensearch
opensearch-node1 | Basedir: /usr/share/opensearch
opensearch-node1 | OpenSearch install type: rpm/deb on Amazon Linux release 2023 (Amazon Linux)
opensearch-node1 | OpenSearch config dir: /usr/share/opensearch/config
opensearch-node1 | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch-node1 | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch-node1 | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch-node1 | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch-node1 | Detected OpenSearch Version: x-content-2.10.0
opensearch-node1 | Detected OpenSearch Security Version: 2.10.0.0
opensearch-node2 | OpenSearch install type: rpm/deb on Amazon Linux release 2023 (Amazon Linux)
opensearch-node2 | OpenSearch config dir: /usr/share/opensearch/config
opensearch-node2 | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch-node2 | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch-node2 | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch-node2 | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch-node2 | Detected OpenSearch Version: x-content-2.10.0
opensearch-node2 | Detected OpenSearch Security Version: 2.10.0.0
opensearch-node1 | /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit.
opensearch-node2 | /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit.
opensearch-node2 | Enabling OpenSearch Security Plugin
opensearch-node1 | Enabling OpenSearch Security Plugin
opensearch-node2 | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
opensearch-node1 | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
opensearch-node2 | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node2 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node2 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
opensearch-node2 | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node1 | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node1 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node1 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
opensearch-node1 | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node1 | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node1 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node1 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
opensearch-node1 | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node2 | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node2 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
...
opensearch-node2 | at org.opensearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:309) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:220) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:118) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node2 | at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:73) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:53) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node2 | at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node2 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:463) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.client.support.AbstractClient.index(AbstractClient.java:487) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2 | at org.opensearch.security.support.ConfigHelper.lambda$uploadFile$0(ConfigHelper.java:97) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2 | at java.security.AccessController.doPrivileged(AccessController.java:569) ~[?:?]
opensearch-node2 | at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:86) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2 | at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:65) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2 | at org.opensearch.security.configuration.ConfigurationRepository.lambda$new$0(ConfigurationRepository.java:146) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2 | at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node1 | [2023-10-11T10:18:56,679][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2 | [2023-10-11T10:18:56,677][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,687][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,688][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,689][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,690][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,690][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:56,709][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node1 | [2023-10-11T10:18:56,767][ERROR][o.o.s.c.ConfigurationRepository] [opensearch-node1] Cannot apply default config (this is maybe not an error!)
opensearch-node1 | org.opensearch.cluster.block.ClusterBlockException: index [.opendistro_security] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];
opensearch-node1 | at org.opensearch.cluster.block.ClusterBlocks.indicesBlockedException(ClusterBlocks.java:243) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.checkBlock(TransportAutoPutMappingAction.java:106) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.checkBlock(TransportAutoPutMappingAction.java:58) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.doExecute(TransportAutoPutMappingAction.java:100) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.doExecute(TransportAutoPutMappingAction.java:58) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node1 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node1 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node1 | at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node1 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node1 | at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.cluster.action.index.MappingUpdatedAction.sendUpdateMapping(MappingUpdatedAction.java:161) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.cluster.action.index.MappingUpdatedAction.updateMappingOnClusterManager(MappingUpdatedAction.java:126) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.bulk.TransportShardBulkAction.lambda$dispatchedShardOperationOnPrimary$1(TransportShardBulkAction.java:414) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:644) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:466) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
opensearch-node1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
opensearch-node1 | at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node1 | [2023-10-11T10:18:56,811][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2 | [2023-10-11T10:18:56,818][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node1 | [2023-10-11T10:18:56,831][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node1 | [2023-10-11T10:18:56,832][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node1 | [2023-10-11T10:18:56,832][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node1 | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node1 | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node1 | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)
opensearch-node2 | [2023-10-11T10:18:57,301][INFO ][o.o.c.r.a.AllocationService] [opensearch-node2] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.opensearch-sap-log-types-config][0]]]).
opensearch-node1 | [2023-10-11T10:18:57,396][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2 | [2023-10-11T10:18:57,401][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node2 | [2023-10-11T10:18:57,636][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:18:57,736][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2 | [2023-10-11T10:18:57,743][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:18:57,744][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards | {"type":"log","@timestamp":"2023-10-11T10:18:57Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2 | [2023-10-11T10:19:00,145][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:19:00,159][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2 | [2023-10-11T10:19:00,162][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:19:00,165][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards | {"type":"log","@timestamp":"2023-10-11T10:19:00Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2 | [2023-10-11T10:19:02,658][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:19:02,676][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2 | [2023-10-11T10:19:02,681][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1 | [2023-10-11T10:19:02,683][ERROR][o.o.s.a.BackendRegistry ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards | {"type":"log","@timestamp":"2023-10-11T10:19:02Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}