ECONNREFUSED when installing opensearch

Hi! I am new to opensearch!

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser):

  • OpenSearch/ Detected OpenSearch Version: x-content-2.10.0 (from the logs with docker logs opensearch-node1)
  • Dashboard/ wasn’t able to find it with docker logs opensearch-dashboards
  • Server OS/ MacOS Big Sur
  • Browser: NaN

Describe the issue:

I am trying to install OpenSearch from the official documentation but I have several problems and warnings. The errors are OpenSearch connection errors: OpenSearch Dashboards cannot connect to OpenSearch due to a connection error (ECONNREFUSED). This could be due to a problem in the configuration.

  • I don’t think it is due to the fact that OpenSearch is not running because it must initialize after it is installed.
  • I don’t think it is because of the RAM because it is indicated Memory: 4 GB
  • I don’t think it is a problem in the configuration because the docker-compose.yaml comes from the official documentation.

Here are the warnings, if they may be the reason:

  • Obsolete tool warning: a warning is displayed informing that a tool will be discontinued in future versions of OpenSearch.A link is provided for more information. Warning about use on public systems: Warns that the security installer should not be used on public or production systems.

  • Security Warning: Warning about the use of an obsolete method in java.lang.System. It is suggested to report this issue to the OpenSearch maintainers, so, here I am!

  • Insecure directory permissions warnings: Warnings issued about insecure permissions on a directory in the OpenSearch configuration. This could pose a security risk?

Configuration:

The one provided here:

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-node2:
    image: opensearchproject/opensearch:latest
    container_name: opensearch-node2
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node2
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch-data2:/usr/share/opensearch/data
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:latest
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
    networks:
      - opensearch-net

volumes:
  opensearch-data1:
  opensearch-data2:

networks:
  opensearch-net:

Relevant Logs or Screenshots:

Not sure what is a relevant log …

(base) remplacement@remplacements-MacBook-Pro Downloads % docker-compose up
[+] Running 3/0
 ✔ Container opensearch-dashboards  C...                                   0.0s 
 ✔ Container opensearch-node2       Create...                              0.0s 
 ✔ Container opensearch-node1       Create...                              0.0s 
Attaching to opensearch-dashboards, opensearch-node1, opensearch-node2
opensearch-node1       | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch-node2       | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch-node1       | **************************************************************************
opensearch-node1       | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch-node1       | ** https://github.com/opensearch-project/security/issues/1755           **
opensearch-node1       | **************************************************************************
opensearch-node2       | **************************************************************************
opensearch-node2       | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch-node2       | ** https://github.com/opensearch-project/security/issues/1755           **
opensearch-node2       | **************************************************************************
opensearch-node1       | OpenSearch Security Demo Installer
opensearch-node1       |  ** Warning: Do not use on production or public reachable systems **
opensearch-node2       | OpenSearch Security Demo Installer
opensearch-node2       |  ** Warning: Do not use on production or public reachable systems **
opensearch-node2       | Basedir: /usr/share/opensearch
opensearch-node1       | Basedir: /usr/share/opensearch
opensearch-node1       | OpenSearch install type: rpm/deb on Amazon Linux release 2023 (Amazon Linux)
opensearch-node1       | OpenSearch config dir: /usr/share/opensearch/config
opensearch-node1       | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch-node1       | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch-node1       | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch-node1       | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch-node1       | Detected OpenSearch Version: x-content-2.10.0
opensearch-node1       | Detected OpenSearch Security Version: 2.10.0.0
opensearch-node2       | OpenSearch install type: rpm/deb on Amazon Linux release 2023 (Amazon Linux)
opensearch-node2       | OpenSearch config dir: /usr/share/opensearch/config
opensearch-node2       | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch-node2       | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch-node2       | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch-node2       | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch-node2       | Detected OpenSearch Version: x-content-2.10.0
opensearch-node2       | Detected OpenSearch Security Version: 2.10.0.0
opensearch-node1       | /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit.
opensearch-node2       | /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit.
opensearch-node2       | Enabling OpenSearch Security Plugin
opensearch-node1       | Enabling OpenSearch Security Plugin
opensearch-node2       | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
opensearch-node1       | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
opensearch-node2       | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node2       | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node2       | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
opensearch-node2       | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node1       | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node1       | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node1       | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
opensearch-node1       | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node1       | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node1       | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
opensearch-node1       | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
opensearch-node1       | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node2       | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node2       | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.10.0.jar)
    ...
opensearch-node2       |    at org.opensearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:309) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:220) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:118) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node2       |    at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:73) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.bulk.TransportSingleItemBulkWriteAction.doExecute(TransportSingleItemBulkWriteAction.java:53) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node2       |    at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node2       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:463) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.client.support.AbstractClient.index(AbstractClient.java:487) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node2       |    at org.opensearch.security.support.ConfigHelper.lambda$uploadFile$0(ConfigHelper.java:97) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2       |    at java.security.AccessController.doPrivileged(AccessController.java:569) ~[?:?]
opensearch-node2       |    at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:86) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2       |    at org.opensearch.security.support.ConfigHelper.uploadFile(ConfigHelper.java:65) ~[opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2       |    at org.opensearch.security.configuration.ConfigurationRepository.lambda$new$0(ConfigurationRepository.java:146) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node2       |    at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node1       | [2023-10-11T10:18:56,679][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2       | [2023-10-11T10:18:56,677][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,687][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,688][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,689][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,690][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,690][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:56,709][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node1       | [2023-10-11T10:18:56,767][ERROR][o.o.s.c.ConfigurationRepository] [opensearch-node1] Cannot apply default config (this is maybe not an error!)
opensearch-node1       | org.opensearch.cluster.block.ClusterBlockException: index [.opendistro_security] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];
opensearch-node1       |    at org.opensearch.cluster.block.ClusterBlocks.indicesBlockedException(ClusterBlocks.java:243) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.checkBlock(TransportAutoPutMappingAction.java:106) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.checkBlock(TransportAutoPutMappingAction.java:58) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.doExecute(TransportAutoPutMappingAction.java:100) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.admin.indices.mapping.put.TransportAutoPutMappingAction.doExecute(TransportAutoPutMappingAction.java:58) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) ~[?:?]
opensearch-node1       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) ~[?:?]
opensearch-node1       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:290) ~[?:?]
opensearch-node1       |    at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) ~[?:?]
opensearch-node1       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) ~[?:?]
opensearch-node1       |    at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.cluster.action.index.MappingUpdatedAction.sendUpdateMapping(MappingUpdatedAction.java:161) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.cluster.action.index.MappingUpdatedAction.updateMappingOnClusterManager(MappingUpdatedAction.java:126) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.bulk.TransportShardBulkAction.lambda$dispatchedShardOperationOnPrimary$1(TransportShardBulkAction.java:414) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:644) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:466) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
opensearch-node1       |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
opensearch-node1       |    at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node1       | [2023-10-11T10:18:56,811][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2       | [2023-10-11T10:18:56,818][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node1       | [2023-10-11T10:18:56,831][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-11T10:18:56,832][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-11T10:18:56,832][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-11T10:18:56,833][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-11T10:18:57,301][INFO ][o.o.c.r.a.AllocationService] [opensearch-node2] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.opensearch-sap-log-types-config][0]]]).
opensearch-node1       | [2023-10-11T10:18:57,396][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node1] Detected cluster change event for destination migration
opensearch-node2       | [2023-10-11T10:18:57,401][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [opensearch-node2] Detected cluster change event for destination migration
opensearch-node2       | [2023-10-11T10:18:57,636][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:18:57,736][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-11T10:18:57,743][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:18:57,744][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-11T10:18:57Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-11T10:19:00,145][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:19:00,159][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-11T10:19:00,162][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:19:00,165][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-11T10:19:00Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-11T10:19:02,658][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:19:02,676][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-11T10:19:02,681][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-11T10:19:02,683][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-11T10:19:02Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}

@AlphaBravo These errors are expected as the OpenSearch Dashboards container is trying to contact with OpenSearch node. However, OpenSerach nodes are not ready yet as the security plugin hasn’t been initialized.

Once the plugin is initialized on both nodes and they communicate with each other, the API layer will be accessible at port 9200. Then OpenSearch Dashboards will successfully connect to one of the OpenSearch nodes.

How long did you wait? Sometimes it can take a minute or two.

@AlphaBravo I’ve just retested the docker-compose from the OpenSearch’s documentation and all worked.

You’ll eventually see the following events in the docker-compose logs.

opensearch-node1         | [2023-10-11T14:34:36,735][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-node1] Node 'opensearch-node1' initialized
...
opensearch-node2         | [2023-10-11T14:34:36,751][INFO ][o.o.s.c.ConfigurationRepository] [opensearch-node2] Node 'opensearch-node2' initialized
...
opensearch-dashboards    | {"type":"log","@timestamp":"2023-10-11T14:34:38Z","tags":["info","savedobjects-service"],"pid":1,"message":"Starting saved objects migrations"}
...
opensearch-dashboards    | {"type":"log","@timestamp":"2023-10-11T14:34:39Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
opensearch-dashboards    | {"type":"log","@timestamp":"2023-10-11T14:34:39Z","tags":["info","http","server","OpenSearchDashboards"],"pid":1,"message":"http server running at http://0.0.0.0:5601"}

Czesc, thanks for your answer @pablo

So I have now waited for more than an hour and most my logs are a a series of error messages related to OpenSearch and its security configuration. It looks like error messages indicate that the OpenSearch nodes and the OpenSearch Dashboards are having trouble initializing, specifically with the security component.

opensearch-node2       | [2023-10-12T09:25:14,049][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:14,074][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T09:25:14,088][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:14,092][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T09:25:14Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-12T09:25:16,550][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:16,560][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T09:25:16,564][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:16,567][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T09:25:16Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-12T09:25:19,042][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:19,050][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T09:25:19,055][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:19,056][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T09:25:19Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-12T09:25:21,536][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:21,542][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T09:25:21,545][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:21,548][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T09:25:21Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-12T09:25:24,093][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:24,100][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T09:25:24,105][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T09:25:24,107][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T09:25:24Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node2       | [2023-10-12T09:25:25,744][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node2] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for internalusers while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for actiongroups while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for config while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for roles while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for rolesmapping while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)
opensearch-node1       | [2023-10-12T09:25:26,534][WARN ][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-node1] No data for tenants while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT]  (index=.opendistro_security)

What is the securityAdmin tool?

@AlphaBravo The securityadmin.sh tool is used to initiate and update the security plugin configuration.

Have you tried to run docker-compose down -v and then start it again with docker-compose up --build?

The docker-compose down -v will delete all the volumes attached to the containers in this docker-compose. It is safe to use in POC/test environment but not in production.

Yes @pablo and unfortunately it looks like it gives me the same errors:

opensearch-node2       | [2023-10-12T11:03:27,167][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,184][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:27,193][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,196][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:27Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node1       | [2023-10-12T11:03:27,298][ERROR][o.o.m.i.MLIndicesHandler ] [opensearch-node1] Failed to create index .plugins-ml-config
opensearch-node1       | org.opensearch.cluster.block.IndexCreateBlockException: blocked by: [FORBIDDEN/10/cluster create-index blocked (api)];
opensearch-node1       | 	at org.opensearch.cluster.block.ClusterBlocks.createIndexBlockedException(ClusterBlocks.java:210) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:98) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:56) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:79) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:320) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) [opensearch-performance-analyzer-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.create(AbstractClient.java:1627) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLIndexIfAbsent(MLIndicesHandler.java:88) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLConfigIndex(MLIndicesHandler.java:66) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.initMLConfig(MLSyncUpCron.java:171) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.run(MLSyncUpCron.java:85) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.threadpool.Scheduler$ReschedulingRunnable.doRun(Scheduler.java:239) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
opensearch-node1       | 	at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node2       | [2023-10-12T11:03:29,638][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,642][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:29,647][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,651][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:29Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}

Yes @pablo and it gives me the same errors:

opensearch-node2       | [2023-10-12T11:03:27,167][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,184][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:27,193][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,196][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:27Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node1       | [2023-10-12T11:03:27,298][ERROR][o.o.m.i.MLIndicesHandler ] [opensearch-node1] Failed to create index .plugins-ml-config
opensearch-node1       | org.opensearch.cluster.block.IndexCreateBlockException: blocked by: [FORBIDDEN/10/cluster create-index blocked (api)];
opensearch-node1       | 	at org.opensearch.cluster.block.ClusterBlocks.createIndexBlockedException(ClusterBlocks.java:210) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:98) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:56) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:79) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:320) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) [opensearch-performance-analyzer-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.create(AbstractClient.java:1627) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLIndexIfAbsent(MLIndicesHandler.java:88) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLConfigIndex(MLIndicesHandler.java:66) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.initMLConfig(MLSyncUpCron.java:171) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.run(MLSyncUpCron.java:85) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.threadpool.Scheduler$ReschedulingRunnable.doRun(Scheduler.java:239) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
opensearch-node1       | 	at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node2       | [2023-10-12T11:03:29,638][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,642][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:29,647][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,651][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:29Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}

Yes @pablo , and it gave me the same issue:

opensearch-node2       | [2023-10-12T11:03:27,167][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,184][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:27,193][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,196][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:27Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node1       | [2023-10-12T11:03:27,298][ERROR][o.o.m.i.MLIndicesHandler ] [opensearch-node1] Failed to create index .plugins-ml-config
opensearch-node1       | org.opensearch.cluster.block.IndexCreateBlockException: blocked by: [FORBIDDEN/10/cluster create-index blocked (api)];
opensearch-node1       | 	at org.opensearch.cluster.block.ClusterBlocks.createIndexBlockedException(ClusterBlocks.java:210) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:98) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:56) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:79) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:320) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) [opensearch-performance-analyzer-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.create(AbstractClient.java:1627) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLIndexIfAbsent(MLIndicesHandler.java:88) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLConfigIndex(MLIndicesHandler.java:66) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.initMLConfig(MLSyncUpCron.java:171) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.run(MLSyncUpCron.java:85) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.threadpool.Scheduler$ReschedulingRunnable.doRun(Scheduler.java:239) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
opensearch-node1       | 	at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node2       | [2023-10-12T11:03:29,638][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,642][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:29,647][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,651][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:29Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}

Yes, it doesn’t change. I get:

opensearch-node2       | [2023-10-12T11:03:27,167][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,184][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:27,193][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:27,196][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:27Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}
opensearch-node1       | [2023-10-12T11:03:27,298][ERROR][o.o.m.i.MLIndicesHandler ] [opensearch-node1] Failed to create index .plugins-ml-config
opensearch-node1       | org.opensearch.cluster.block.IndexCreateBlockException: blocked by: [FORBIDDEN/10/cluster create-index blocked (api)];
opensearch-node1       | 	at org.opensearch.cluster.block.ClusterBlocks.createIndexBlockedException(ClusterBlocks.java:210) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:98) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:56) ~[opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.doStart(TransportClusterManagerNodeAction.java:237) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction$AsyncSingleAction.tryAction(TransportClusterManagerNodeAction.java:206) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction$1.doRun(RetryableAction.java:137) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.RetryableAction.run(RetryableAction.java:115) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:167) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.clustermanager.TransportClusterManagerNodeAction.doExecute(TransportClusterManagerNodeAction.java:79) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.controlcenter.notification.filter.IndexOperationActionFilter.apply(IndexOperationActionFilter.kt:39) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) [opensearch-index-management-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:320) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:165) [opensearch-security-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) [opensearch-performance-analyzer-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:216) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1537) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.client.support.AbstractClient$IndicesAdmin.create(AbstractClient.java:1627) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLIndexIfAbsent(MLIndicesHandler.java:88) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.indices.MLIndicesHandler.initMLConfigIndex(MLIndicesHandler.java:66) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.initMLConfig(MLSyncUpCron.java:171) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.ml.cluster.MLSyncUpCron.run(MLSyncUpCron.java:85) [opensearch-ml-2.10.0.0.jar:2.10.0.0]
opensearch-node1       | 	at org.opensearch.threadpool.Scheduler$ReschedulingRunnable.doRun(Scheduler.java:239) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.10.0.jar:2.10.0]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
opensearch-node1       | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
opensearch-node1       | 	at java.lang.Thread.run(Thread.java:833) [?:?]
opensearch-node2       | [2023-10-12T11:03:29,638][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,642][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-node2       | [2023-10-12T11:03:29,647][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node2] Not yet initialized (you may need to run securityadmin)
opensearch-node1       | [2023-10-12T11:03:29,651][ERROR][o.o.s.a.BackendRegistry  ] [opensearch-node1] Not yet initialized (you may need to run securityadmin)
opensearch-dashboards  | {"type":"log","@timestamp":"2023-10-12T11:03:29Z","tags":["error","opensearch","data"],"pid":1,"message":"[ResponseError]: Response Error"}

@AlphaBravo Have you tried running the securityadmin.sh script against the running cluster?
Also, could you share the environment variables (env command) of the docker host?
What version of the docker and Linux distro do you use?

It’s now all sorted, sorry for the message flooding I was put on hold by the website for being a newcomer here.
Thanks for all your help !

Hi @Charlie , how did you solve this issue ?
thank you

@julien1 If you need help with your issue, please open a new thread in the forum and share all the problem details.